User Access Control
DolphinDB offers a powerful, flexible, and secure access control system with the following features:
- Access roles like admin, user, and group for convenient permission control.
- 19 types of access privileges for various scenarios.
- Built-in access control functions/commands.
- Function views to provide analysis results without sacrificing data privacy.
- Dynamic access control for scheduled jobs and streaming tasks.
- RSA encryption for key information.
- SSO to simplify cross-domain access.
This tutorial will cover the key components of DolphinDB's access control system and how to effectively manage database security.
1. User Roles
Before logging in DolphinDB, a user can access databases as a guest who does not have (and cannot be granted with) any access privileges.
A user can log in (out) with command login
(logout
).
1.1 User Types
1.1.1 Users and Groups
DolphinDB uses groups to manage users with the access privileges. A user can belong to 0, 1 or multiple groups, and a group can have 0, 1 or multiple users.
A user's access privileges are determined by his/her own privileges and the privileges of the groups that he/she belongs to. You can grant or deny the permissions for users or groups. When the permission of a group changes, it applies to all members in the group. When a new member joins a group, the user inherits all privileges from the group.
1.1.2 Administrators
There are 2 types of administrators in DolphinDB: super admin and admin.
When a DolphinDB cluster is started for the first time, a super admin is created with user ID "admin" and password "123456". This super admin cannot be deleted and has all access privileges that cannot be revoked or denied.
To create an admin, you can set the parameter isAdmin to true for function createUser
.
1.1.3 Administrators v.s. Users
Users other than the administrators are regular users.
Only administrators can create/delete users and groups, and grant
/revoke
/deny
the permissions of other administrators, users and groups. The following user-management functions can only be executed by administrators: resetPwd
, createUser
, deleteUser
, createGroup
, deleteGroup
, addGroupMember
, and deleteGroupMember
.
Users (including administrators) can change their passwords by calling changePwd
. Administrators can use resetPwd
to modify the passwords for other users.
The following table summarizes the differences among super admin, admin and regular user.
super admin | admin | user | |
---|---|---|---|
Need to be manually created | × | √ | √ |
Initial privileges | All Privileges | None | None |
Can be deleted | × | √ | √ |
Can be listed by getUserList() | × | √ | √ |
Can create or delete admin/user/group | √ | √ | × |
Can grant or deny privileges for admin/user/group | √ | √ | × |
Can create or delete function views | √ | √ | × |
Can delete scheduled jobs submitted by other users | √ | √ | × |
1.2 Role Management
As of version 1.30.21/2.00.9, user-management functions can be executed on data nodes, while in previous versions these functions can only be called on controllers.
1.2.1 User Management
Only administrators can create users:
login(`admin, "123456")
createUser("admin1","123456",,true);
login("admin1","123456")
createUser("user1","123456",,false) // create a user
createUser("user2","123456",,true) // create an admin
All users can change their own passwords:
//user1 changes password
login("user1","123456")
changePwd("123456","123456@")
logout("user1")
login("user1","123456@") // password changed successfully
Only administrators can change other users' passwords:
// admin changes password for user1 login
login("admin1","123456")
resetPwd("user1","123456")
login("user1","123456")
// successfully logged in with new password
Only administrators can delete users:
// admin deletes user1
login("admin1","123456")
deleteUser("user1")
login("user1","123456")
// Error: The user name or password is incorrect
1.2.2 Group Management
Only administrators can manage user groups, including:
Creating a group:
createGroup("group1",["admin1"])
Adding a group member:
addGroupMember(["user2"],"group1")
Removing a group member:
deleteGroupMember(["user2"],"group1")
Deleting a group:
deleteGroup("group1")
When a group is deleted, the group members are not deleted, but all permissions that were applied to the group are revoked. For details on the privileges on users and groups, see Section 3.1 Users and Groups' Access Privileges.
1.2.3 Check User Information
getGroupList
: Return a list of group names. It can only be executed by administrators.getUsersByGroupId
: Return a string vector of the user names that belong to the specified group. It can only be executed by administrators.getGroupsByUserId
: Return a string vector of the groups that the user belongs to. It can only be executed by administrators.getGroupAccess
: Return a table of privileges for one or multiple groups. It can only be executed by administrators.getUserList
: Return a list of user names other than the administrators. It can only be executed by administrators.getUserAccess
: Return privileges for individual users, without taking into account the privileges for the groups the users belong to. When userId is not specified, return the privileges for the current user. Only an administrator can use this function to get other users' privileges.
2. Access Management
Users can use commands grant
, deny
and revoke
to set access privileges, which are marked as "allow", "deny" or "none" in the permission state.
Note: As of version 1.30.21/2.00.9, access-management functions can be executed on data nodes, while in previous versions these functions can only be called on controllers.
grant
: grant specific privileges to a user/group
grant(`user2, DBOBJ_CREATE,"*") getUserAccess("user2")
revoke
: revoke specific privileges that were previously granted or denied to a use/group
revoke(`user2, DBOBJ_CREATE,"*") getUserAccess("user2")
deny
: deny specific privileges to a user/group
deny(`user2, DBOBJ_CREATE,"*") getUserAccess("user2")
3. Privilege Rules
3.1 Users and Groups
For the privilege rules discussed below, a user is viewed as belonging to a special group with himself/herself as the only member. The user's access privileges are determined by the privileges of the groups that he/she belongs to.
Different groups may set different access privileges to a user. For these situations the following rules apply:
- If a user is granted a privilege in one group and this privilege is not denied in any other group, then the user is granted the privilege.
- If a user is denied a privilege in at least one group, then the user's privilege is denied.
Example 1. If a user belongs to one group, and the user's permission state is deny, while the group's state is allow, then the user's permission is denied.
login("admin","123456")
createUser("user1","123456")
createGroup("group1",["user1"])
deny(`user1, DB_OWNER) // deny DB_OWNER to user1
grant(`group1, DB_OWNER) // grant DB_OWNER to group1
login("user1","123456")
database("dfs://test",VALUE,1..10) // user1 is denied
// Error: <NoPrivilege>Not granted to create or delete databases
Example 2. If a user belongs to one group, and the user's permission state is none, while the group's state is allow, then the user's permission is granted.
login("admin","123456")
revoke(`user1, DB_OWNER) // revoke the denied DB_OWNER
grant(`group1, DB_OWNER) // grant DB_OWNER to group1
login("user1","123456")
database("dfs://test",VALUE,1..10) // user1 is granted
Example 3. If a user belongs to one group, and the user's permission state is none, while the group's state is deny, then the user's permission is denied.
login("admin","123456")
revoke(`user1, DB_OWNER) // revoke the denied DB_OWNER
deny(`group1, DB_OWNER) // deny DB_OWNER to group1
login("user1","123456")
database("dfs://test",VALUE,1..10)// user1 is denied
// Error: <NoPrivilege>Not granted to create or delete databases
Example 4. If a user belongs to multiple groups, and some groups' permission state is deny, while some is allow, then the user's permission is denied.
login("admin","123456")
createGroup("group2",["user1"])
createGroup("group3",["user1"])
deny(`group1, DB_OWNER)
grant(`group2, DB_OWNER)
grant(`group3, DB_OWNER)
login("user1","123456")
database("dfs://test",VALUE,1..10) // user1 is denied
// Error: <NoPrivilege>Not granted to create or delete databases
Example 5. If a user belongs to multiple groups, and some groups' permission state is none, while some is allow, then user's permission is granted.
login("admin","123456")
revoke(`group1, DB_OWNER)
grant(`group2, DB_OWNER)
grant(`group3, DB_OWNER)
login("user1","123456")
database("dfs://test",VALUE,1..10) // user1 is granted
Example 6. If a user belongs to multiple groups, and some groups' permission state is none, while some is deny, then the user's permission is denied.
login("admin","123456")
revoke(`group1, DB_OWNER)
deny(`group2, DB_OWNER)
deny(`group3, DB_OWNER)
login("user1","123456")
database("dfs://test",VALUE,1..10) // user1 is denied
// Error: <NoPrivilege>Not granted to create or delete databases
Example 7. If a user belongs to a group, the user retains only his/her own privileges after the group is deleted.
login("admin","123456")
createUser("user1","123456")
createGroup("group1",["user1"])
grant(`user1, DB_OWNER) //grant DB_OWNER to user1
deny(`group1, DB_OWNER) //deny DB_OWNER to group1
deleteGroup(`group1) //delete group1
login("user1","123456")
database("dfs://test",VALUE,1..10) //user1 is granted
login("admin","123456")
createGroup("group1",["user1"])
revoke(`user1, DB_OWNER) //revoke granted DB_OWNER
grant(`group1, DB_OWNER) //grant DB_OWNER to group1
deleteGroup(`group1) //delete group1
login("user1","123456")
database("dfs://test",VALUE,1..10) //user1 is denied
3.2 Access Scopes
Based on the application scope, permissions are divided into three scopes: global (*), database and table.
The following are privilege rules for determining permissions on databases and tables:
- If you
grant/revoke/deny
at a smaller scope, and then manage permissions at a larger scope, the smaller scope permissions will be invalidated and overridden by the larger scope permissions. - If you
allow
at a larger scope, and thendeny
at a smaller scope, thedeny
operation will take effect while theallow
permissions still apply to the objects outside of the deny scope. - If you
allow
/deny
at a larger scope, and thenrevoke
at a smaller scope, therevoke
operation will be invalid while the allow permissions still apply to the original scope. - If you
deny
at a larger scope, and thengrant
at a smaller scope:- For versions before 1.30.21 and 2.00.9, the
grant
can take effect while thedeny
permissions still apply to the objects outside of the grant scope. - Since version 1.30.21 and 2.00.9, this operation is not allowed.
- For versions before 1.30.21 and 2.00.9, the
- If you
revoke
in a large scope, and then manage permissions at a smaller scope, the smaller scope permissions will take effect.
Example 1. First deny
TABLE_READ at table level to user 1, then grant
TABLE_READ at global level. Now user1 has the TABLE_READ access to all tables:
login("admin", "123456")
createUser("user1","123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "id")
pt.append!(t)
deny("user1", TABLE_READ, dbName+"/pt")
grant("user1", TABLE_READ, "*")
login("user1", "123456")
select * from loadTable(dbName, "pt")
// user1 is granted TABLE_READ to "dfs://test"
Example 2. First grant
TABLE_READ at table level to user2, then deny
TABLE_READ at global level. Now user2 is denied to access all tables:
login("admin", "123456")
createUser("user2","123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "id")
pt.append!(t)
grant("user2", TABLE_READ, dbName+"/pt")
deny("user2", TABLE_READ, "*")
login("user2", "123456")
select * from loadTable(dbName, "pt")
// user2 is denied to access "dfs://test"
Example 3. First grant
TABLE_READ at table level to user3, then revoke
TABLE_READ at global level. Now the TABLE_READ permissions on all tables of user3 are revoked.
login("admin", "123456")
createUser("user3","123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "id")
pt.append!(t)
grant("user3", TABLE_READ, dbName+"/pt")
revoke("user3", TABLE_READ, "*")
login("user3", "123456")
select * from loadTable(dbName, "pt")
// user3's privilege is revoked
Example 4. First grant
TABLE_READ at global level to user1, then deny
TABLE_READ on the table "dfs://test/pt" to user1. Now user1 is granted TABLE_READ access to all tables except "dfs://test/pt".
login("admin", "123456")
createUser("user1","123456")
dbName = "dfs://test"
t = table(1..10 as id , rand(100, 10) as val)
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "id")
pt.append!(t)
grant("user1", TABLE_READ, "*")
deny("user1", TABLE_READ, dbName+"/pt")
login("user1", "123456")
select * from loadTable(dbName, "pt")
// user1's access to "dfs://test/pt" is denied
Example 5. First grant
TABLE_READ to user1, then revoke
user1's TABLE_READ access to "dfs://test/pt". Now user1 is still granted TABLE_READ access to all tables.
login("admin", "123456")
createUser("user1","123456")
dbName = "dfs://test"
t = table(1..10 as id , rand(100, 10) as val)
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "id")
pt.append!(t)
grant("user1", TABLE_READ, "*")
revoke("user1", TABLE_READ, dbName+"/pt")
login("user1", "123456")
select * from loadTable(dbName, "pt")
// user1 is granted to access "dfs://test/pt"
Example 6. First deny
TABLE_READ at global level to user1, then revoke
the denied TABLE_READ access to table "dfs://test/pt". Now user1 is still denied to read all tables.
login("admin", "123456")
createUser("user1","123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "id")
pt.append!(t)
deny("user1", TABLE_READ, "*")
revoke("user1", TABLE_READ, dbName+"/pt")
getUserAccess("user1")
Example 7. First deny
TABLE_READ at global level to user1, then grant
TABLE_READ on table "dfs://test/pt" to user1.
For versions before 1.30.21/2.00.9, user1 has the TABLE_READ access to table "dfs://test/pt", but the access to other tables is denied; For versions since 1.30.21/2.00.9, the grant
operation will raise an error.
login("admin", "123456")
createUser("user1","123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "id")
pt1= db.createPartitionedTable(t, "pt1", "id")
pt.append!(t)
pt1.append!(t)
deny("user1", TABLE_READ, "*")
// As of 1.30.21/2.00.9 version, the grant operation reports 'Invalid grant: grant [dfs://test/pt] and [deny *] are in conflict'
grant("user1", TABLE_READ, dbName+"/pt")
login("user1", "123456")
select * from loadTable(dbName, "pt")
// In the previous versions, user1 is granted to access "dfs://test/pt"
select * from loadTable(dbName, "pt1")
// user1 is denied to access other tables
3.3 Compatibility
If users created before version 1.30.21/2.00.9 were granted table-level privileges like TABLE_READ and TABLE_WRITE on some tables while denied those privileges on other tables, after a server upgrade to version 1.30.21/2.00.9 or higher, they have access to all tables except those that were explicitly denied.
For example, user1 is granted TABLE_READ access to pt1, and denied TABLE_READ access to pt2 in versions before 1.30.21/2.00.9. Then user1 is only granted to access "dfs://valuedb/pt1".
login("admin","123456")
createUser("user1","123456")
grant("user1",TABLE_READ,"dfs://valuedb/pt1")
deny("user1",TABLE_READ,"dfs://valuedb/pt2")
After the server is upgraded, user1 is granted to access to all tables of database "dfs://valuedb" except pt2. The above script is equivalent to:
grant("user1",TABLE_READ,"dfs://valuedb/*")
deny("user1",TABLE_READ,"dfs://valuedb/pt2")
4. Privilege Types
4.1 Database-Level Privileges
Privilege | Description |
---|---|
DB_MANAGE | Database level: delete databases. Table level: create/delete/rename tables, add/delete partitions, add/delete/rename/replace columns |
DB_OWNER | Database level: create databases; delete databases created by themselves. Table level (in databases owned by themselves): create/delete/rename tables, add/delete partitions, add/delete columns. Users with this privilege can grant other users the following privileges on the databases created by themselves: TABLE_READ, TABLE_WRITE, TABLE_INSERT, TABLE_UPDATE, TABLE_DELETE, DBOBJ_CREATE, DBOBJ_DELETE, DB_READ, DB_WRITE, DB_INSERT, DB_UPDATE, DB_DELETE |
DBOBJ_CREATE | create tables in specific databases |
DBOBJ_DELETE | delete tables and schema from specific databases |
DB_READ | read tables in specific databases |
DB_WRITE | write to tables in specific databases |
DB_INSERT | insert into tables in specific databases |
DB_UPDATE | update tables in specific databases |
DB_DELETE | delete tables from specific databases |
4.2 Table-Level Privileges
Privilege | Description |
---|---|
TABLE_READ | read tables |
TABLE_WRITE | write to tables |
TABLE_INSERT | insert into tables |
TABLE_UPDATE | update tables |
TABLE_DELETE | delete tables |
4.3 Other Privileges
Privilege | Description |
---|---|
VIEW_EXEC | execute specific function views |
SCRIPT_EXEC | execute scripts |
TEST_EXEC | execute test scripts |
QUERY_RESULT_MEM_LIMIT | set the memory limit for a query result |
TASK_GROUP_MEM_LIMIT | set the memory limit for a task group |
4.4 Compatibility of Privilege Types
(1) If users created before version 1.30.21/2.00.9 were only granted DB_MANAGE but not DB_OWNER, after a server upgrade they are not allowed to create databases.
(2) If users created before version 1.30.21/2.00.9 were granted TABLE_WRITE, after a server upgrade they are also granted TABLE_INSERT, TABLE_UPDATE, and TABLE_DELETE privileges.
(3) When using grant/deny/revoke
with accessType=DB_OWNER, the objs parameter for databases must end with "*", indicating the prefix pattern of databases.
login("admin", "123456")
createUser("user1","123456")
grant("user1", DB_OWNER,"*")
// user1 is granted DB_OWNER access to all databases
grant("user1", DB_OWNER,"dfs://test0*")
// user1 is granted DB_OWNER access to all databases with a predix of "dfs://test0"
(4) When using grant/deny/revoke
with accessType=DB_MANAGE, the databases specified in objs must already be created. Otherwise an error will be reported.
login("admin", "123456")
createUser("user1","123456")
grant("user1", DB_MANAGE,"dfs://test0")
// Error: The database [dfs://test0] does not exist
grant("user1", DB_MANAGE, ["dfs://db1","dfs://db2"])
// Error: The database [dfs://db1] does not exist
4.5 Privilege Revocation
A table-level privilege is revoked when the involved user or table is deleted, and a database-level privilege is revoked when the involved user or database is deleted. A DB_OWNER privilege is revoked only when the user and all involved databases are deleted.
For privileges applied on shared in-memory tables or streaming engines, they will not be revoked after server restarts, or when the shared tables or streaming engines are deleted. Therefore, if new shared tables or streaming engines are created with the same names, the previously-assigned privileges still take effect.
Example 1. user1 is granted TABLE_READ on table "dfs://valuedb/pt". Delete and recreate the table, and the TABLE_READ privilege is revoked from user1.
login("admin", "123456")
dbName = "dfs://valuedb"
t = table(1..10 as id , rand(100, 10) as val)
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "id").append!(t)
createUser("user1","123456")
grant("user1", TABLE_READ,"dfs://valuedb/pt")
login("user1", "123456")
select * from loadTable("dfs://valuedb",`pt)
// user1 is granted to access "dfs://valuedb/pt"
login("admin", "123456")
dropTable(db,`pt)
pt= db.createPartitionedTable(t, "pt", "id").append!(t)
login("user1", "123456")
select * from loadTable("dfs://valuedb",`pt)
// the TABLE_READ access of user1 is revoked
Example 2. user1 is granted DB_MANAGE on the database "dfs://valuedb". Delete and recreate the database, and the DB_MANAGE privilege is revoked from user1.
login("admin", "123456")
dbName = "dfs://valuedb"
t = table(1..10 as id , rand(100, 10) as val)
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "id").append!(t)
createUser("user1","123456")
grant("user1", DB_MANAGE,"dfs://valuedb")
getUserAccess("user1")
Now user1 is granted the DB_MANAGE access to database "dfs://valuedb".
login("admin", "123456")
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "id").append!(t)
getUserAccess("user1")
The DB_MANAGE privilege is revoked from user1.
Example 3. The user u1 is granted read and write privileges on the stream tables "trades", "output1" and the streaming engine "agg1":
login(`admin, `123456)
createUser(`u1, "111111");
enableTableShareAndPersistence(streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]),`trades)
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as output1
agg1 = createTimeSeriesEngine(name="agg1", windowSize=600, step=600, metrics=<[sum(volume)]>, dummyTable=trades, outputTable=output1, timeColumn=`time, useSystemTime=false, keyColumn=`sym, garbageSize=50, useWindowStartTime=false)
grant("u1", TABLE_READ, "agg1")
grant("u1", TABLE_WRITE, "agg1")
grant("u1", TABLE_READ, "trades")
grant("u1", TABLE_WRITE, "trades")
grant("u1", TABLE_READ, "output1")
grant("u1", TABLE_WRITE, "output1")
getUserAccess(`u1)
If server restarts, or a user deletes and recreates the stream table "trades", "output1" or the streaming engine "agg1", access control is not applied to these objects, and any user can access them.
login(`admin, `123456)
enableTableShareAndPersistence(streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]),`trades)
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as output1
agg1 = createTimeSeriesEngine(name="agg1", windowSize=600, step=600, metrics=<[sum(volume)]>, dummyTable=trades, outputTable=output1, timeColumn=`time, useSystemTime=false, keyColumn=`sym, garbageSize=50, useWindowStartTime=false)
Add access control to these objects:
addAccessControl(`trades)
addAccessControl(output1)
addAccessControl(agg1)
Now only u1 and the creator admin can access these objects:
login(`u1, "111111")
insert into agg1 values(2018.10.08T01:01:01.785,`A,10)
insert into output1 values(2018.10.08T01:01:01.785,`A,10)
insert into trades values(2018.10.08T01:01:01.785,`A,10)
Other users are denied:
login(`u2, "111111")
insert into agg1 values(2018.10.08T01:01:01.785,`A,10)=> No access to table [agg1]'
insert into output1 values(2018.10.08T01:01:01.785,`A,10) => No access to table [output1]'
insert into trades values(2018.10.08T01:01:01.785,`A,10) => No access to table [trades]'
5. Privilege Verification
5.1 Privileges Required by DDL/DML Operations
The following table introduces the privileges required by DDL/DML operations before and since version 1.30.21/2.00.9.
Operation | Required Privileges (Since version 1.30.21/2.00.9) | Required Privileges (Before version 1.30.21/2.00.9) |
---|---|---|
database (create databases) | DB_OWNER to create databases at global level or databases with specific prefix | DB_MANAGE or DB_OWNER to create databases |
database (load databases) | None | None |
dropDatabase | DB_MANAGE to delete any database;DB_OWNER to delete databases created by the user | DB_MANAGE to delete any database;DB_OWNER to delete databases created by the user |
createTable | DB_MANAGE/DBOBJ_CREATE to create tables in any database;DB_OWNER to create tables in databases created by the user | DBOBJ_CREATE to create tables in any database;A user who creates the database can create tables in that database |
dropTable | DB_MANAGE/DBOBJ_DELETE to delete tables in any database;DB_OWNER to delete tables in databases created by the user | DB_OWNER to delete any table in databases created by the user;DB_MANAGE to delete any table created by the user;A user with DBOBJ_DELETE privilege on the database can delete any table in that database |
createPartitionedTable | DB_MANAGE/DBOBJ_CREATE to create tables in any database; DB_OWNER to create tables in databases created by the user | DBOBJ_CREATE to create tables in any database;A user who creates the database can create tables in that database |
addPartitions(addValuePartitions/addRangePartitions) | DB_MANAGE to add partitions to any database;DB_OWNER to add partitions to databases created by the user | DB_MANAGE to add partitions to any database;DB_OWNER to add partitions to databases created by the user |
configure newValuePartitionPolicy=add | The parameter is automatically added once a new partition is written by a user with write privilege. | The parameter is automatically added once a new partition is written by a user with write privilege. |
dropPartition (deleteSchema=false) | DB_MANAGE/DB_DELETE to delete partitions from any database;DB_OWNER to delete partitions from databases created by the user;TABLE_DELETE at global level to delete partitions from any database | DB_OWNER to delete partitions from databases created by the user; DBOBJ_DELETE to delete partitions from any database |
dropPartition(deleteSchema=true) | DB_MANAGE to delete partitions from any database; DB_OWNER to delete partitions from databases created by the user | DB_OWNER to delete partitions from databases created by the user;DBOBJ_DELETE to delete partitions from any database |
renameTable | DB_MANAGE/DBOBJ_CREATE to rename tables from any database;DB_OWNER to rename tables in databases created by the user | DB_OWNER to rename tables in databases created by the user; DBOBJ_DELETE to rename tables from specific databases |
addColumn | DB_MANAGE/DBOBJ_CREATE to add columns to tables in any database;DB_OWNER to add columns to tables in databases created by the user | DB_OWNER/DB_MANAGE to add columns to tables in databases created by the user;DBOBJ_CREATE to add columns to tables in specific databases |
dropColumns! | DB_MANAGE/DBOBJ_DELETE to delete columns from any database;DB_OWNER/DBOBJ_CREATE to delete columns from databases created by the user | DB_MANAGE to delete columns from any database; DB_OWNER to delete columns from databases created by the user |
rename! | DB_MANAGE/DBOBJ_CREATE to rename columns in any database; DB_OWNER to rename columns from databases created by the user | DB_MANAGE to rename columns in any database; DB_OWNER to rename columns from databases created by the user |
replaceColumn! | DB_MANAGE/DBOBJ_CREATE to replace columns of tables from any database;DB_OWNER to replace columns of tables from databases created by the user | DB_MANAGE to replace columns of tables from any database; DB_OWNER to replace columns of tables from databases created by the user |
setColumnComment | DB_MANAGE/DBOBJ_CREATE to set column comments for tables in any database;DB_OWNER to set column comments for tables in databases created by the user | DB_OWNER/DB_MANAGE to set column comments for tables in databases created by the user;DBOBJ_CREATE to set column comments for tables in specific databases |
truncate | DBOBJ_CREATE/DB_OWNER/DB_MANAGE to truncate tables from databases created by the user;TABLE_WRITE/TABLE_DELETE/DB_WRITE/DB_DELETE to truncate tables from specific databases;VIEW_EXEC to execute truncate in function views | DBOBJ_CREATE/DB_OWNER/DB_MANAGE to truncate tables from databases created by the user;TABLE_WRITE to truncate tables from specific databases;VIEW_EXEC to execute truncate in function views |
append! | DBOBJ_CREATE/DB_OWNER/DB_MANAGE to append data to tables created by the user;TABLE_WRITE/TABLE_INSERT/DB_WRITE/DB_INSERT to append data to specific tables;VIEW_EXEC to execute append! in function views | DBOBJ_CREATE/DB_OWNER/DB_MANAGE to append data to tables created by the user;TABLE_WRITE to append data to specific tables;VIEW_EXEC to execute append! in function views |
upsert/SQL update | DBOBJ_CREATE/DB_OWNER/DB_MANAGE to upsert or update tables created by the user;TABLE_WRITE/TABLE_UPDATE/DB_DELETE/DB_UPDATE to upsert or update specific tables;VIEW_EXEC to execute upsert /update in function views | DBOBJ_CREATE/DB_OWNER/DB_MANAGE to upsert or update tables created by the user;TABLE_WRITE to upsert or update specific tables;VIEW_EXEC to execute upsert /update in function views |
SQL delete | DBOBJ_CREATE/DB_OWNER/DB_MANAGE to delete tables created by the user; TABLE_WRITE/TABLE_DELETE/DB_WRITE/DB_DELETE to delete specific tables;VIEW_EXEC to execute SQL delete in function views | DBOBJ_CREATE/DB_OWNER/DB_MANAGE to delete tables created by the user; TABLE_WRITE to delete specific tables;VIEW_EXEC to execute SQL delete in function views |
When calling function addColumn
, dropColumns!
, renameColumns
, replaceColumns
, setColumnComment
, rename!
, or renameTable
, if the applied object is created by other users, the current user must be granted TABLE_READ access to the table.
For example:
user 1 creates database "dfs://test" and grants user2 DBOBJ_CREATE. Then log in as user2 to create table pt1.
login("admin", "123456")
createUser("user1","123456",,true)
createUser("user2","123456",,true)
grant(`user1, DB_OWNER,"dfs://test*")
login("user1", "123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
grant(`user2, DBOBJ_CREATE ,"dfs://test")
login("user2", "123456")
pt1= db.createPartitionedTable(t, "pt1",`id).append!(t)
If user1 attempts to rename a column in pt1, error of no privilege is reported:
login("user1", "123456")
rename!(pt1,`val,`val1)
// Error: <NoPrivilege> Not granted to read table dfs://test/pt1'
user1 must be granted the TABLE_READ privilege on pt1:
login("admin", "123456")
grant(`user1, TABLE_READ,"dfs://test/pt1")
Then log in as user1 to re-execute the rename operation:
login("user1", "123456")
rename!(pt1,`val,`val1)
schema(loadTable(dbName,`pt1)).colDefs
------------------------------------------
name typeString typeInt extra comment
---- ---------- ------- ----- -------
id INT 4
val1 INT 4
5.2 Visibility of Functions
Function | Required Privileges (Since version 1.30.21/2.00.9) | Required Privileges (Before version 1.30.21/2.00.9) |
---|---|---|
getAllDBs | Executed by administrators or users with DB_MANAGE privilege to return all DFS databases on the current node; Otherwise, return DFS databases created by the user or databases (to which the user has DB_MANAGE privileges) | return all DFS databases on the current node |
getTables | Return all tables in the specific database | return all tables in the specific database |
getClusterDFSDatabases | Executed by administrators or users with DB_MANAGE privilege to return all DFS databases on the current node; Otherwise, return DFS databases created by the user or databases (to which the user has DB_MANAGE privileges) | return all DFS databases on the current node |
getDFSDatabases | Executed by administrators or users with DB_MANAGE privilege to return all DFS databases on the current node; Otherwise, return DFS databases created by the user or databases (to which the user has DB_MANAGE privileges) | return all DFS databases on the current node |
getDFSTablesByDatabase | Executed by administrators or users with DB_MANAGE privilege to return tables in all DFS databases on the current node; Otherwise, return tables in DFS databases created by the user or databases (to which the user has DB_MANAGE privileges) | return tables in all DFS databases on the current node |
Since version 1.30.21/2.00.9, users with VIEW_EXEC privilege only cannot get expected results when executing the above functions.
6. Privilege Management
This chapter introduces the privilege management in DFS databases, shared in-memory tables, streaming data and tasks, function views, scheduled jobs, and memory management.
6.1 DFS Databases
For DFS databases, you can use function grant/deny/revoke
to set privilege access for users or groups. The parameter accessType of these functions can take all the privilege types.
- Global or table-level privileges: TABLE_READ, TABLE_WRITE, TABLE_INSERT, TABLE_UPDATE, TABLE_DELETE
- Global or database-level privileges: DBOBJ_CREATE, DBOBJ_DELETE, DB_READ, DB_WRITE, DB_INSERT, DB_UPDATE, DB_DELETE, DB_MANAGE, DB_OWNER
Example 1. Grant user1 TABLE_READ at global level:
grant(`user1, TABLE_READ, "*")
Example 2. Grant user1 TABLE_READ of table pt in the database "dfs://valuedb":
grant(`user1, TABLE_READ,"dfs://valuedb/pt")
Example 3. Grant user1 DBOBJ_CREATE in the database "dfs://valuedb":
grant(`user1, DBOBJ_CREATE,"dfs://valuedb")
Example 4. Grant user1 the read and write privileges in a specific database (new feature in version 1.30.21/2.00.9):
login("admin","123456")
createUser("user1","123456")
grant("user1",DB_READ,"dfs://valuedb")
grant("user1",DB_WRITE,"dfs://valuedb")
Now user1 has all access privileges to database "dfs://valuedb".
Example 5. Grant user2 the privilege to create and manage databases with prefix "dfs://db0" only.
login("admin","123456")
createUser("user2","123456")
grant("user2", DB_OWNER, "dfs://db0*")
In this case, if user2 attempts to create a database with prefix "dfs://db1", the operation will be denied.
Example 6. Grant all members of a group the read privilege on a table, and only some members with write privilege.
login("admin","123456")
createUser("user1","123456")
createUser("user2","123456")
createUser("user3","123456")
createGroup("group1",["user1","user2","user3"])
grant("group1", TABLE_READ, "dfs://valuedb/pt")
createUser("user4","123456")
addGroupMember("user4","group1")
grant("user1", TABLE_WRITE, "dfs://valuedb/pt")
grant("user4", TABLE_WRITE, "dfs://valuedb/pt")
Now user1 and user4 are granted read and write privileges on table "dfs://valuedb/pt", and user2 and user3 have only the read privilege.
Example 7. grant
or deny
privileges on all objects (represented by *). Grant user1 the read privilege on all DFS tables:
grant("user1",TABLE_READ,"*")
If the grant or deny operation applies to all objects, revoke
can only be used to cancel the privilege settings from all objects. If only the privilege access on a specific object is revoked, the revoke operation does not take effect:
revoke("user1",TABLE_READ,"dfs://db1/t1")
Instead, use the following command to revoke granted privileges from all objects:
revoke("user1",TABLE_READ,"*")
Similarly, after using grant
or deny
on a group, only revoke
can be used to cancel the privilege settings. Note the revoke
command can only be applied to the group. If only the access of a member is revoked, the operation does not take effect.
Example 8. Grant DB_OWNER to user1. Log in as user1 to create table "dfs://dbMT/dt" and grant user2 the read privilege on the table. Now user2 now can access the table "dfs://dbMT/dt".
createUser("user1", "123456")
createUser(`user2, "123456")
grant("user1",DB_OWNER);
login("user1", "123456")
db = database("dfs://dbMT", VALUE, 1..10)
t=table(1..1000 as id, rand(100, 1000) as x)
dt = db.createTable(t, "dt").append!(t)
grant("user2", TABLE_READ, "dfs://dbMT/dt");
6.2 Shared In-Memory Tables
In DolphinDB, the shared in-memory tables are shared across all sessions, and the read and write permissions are managed in the access system. The access types for shared in-memory tables only support TABLE_READ and TABLE_WRITE.
Note:
- Only the table creator or administrator is allowed to set other users' access privileges on that table.
- If the table creator or administrator calls
addAccessControl
to restrict other users from accessing the shared table, other users can only access the table after being granted access privileges. - If the table creator or administrator has set access privileges on the table for other users with
grant/deny/revoke
, access control will be automatically added. Users without granted privileges cannot access the table.
Examples:
Create an administrator admin1. Log in as admin1 to create a shared in-memory table st1:
login(`admin, "123456");
createUser("admin1","123456",,true)
login(`admin1, "123456")
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as st1
All users have the read and write privileges on the table st1:
createUser("user1","123456",,false)
login("user1","123456")
insert into st1 values(1970.01.01T00:00:00.001,`sym,1)
select * from st1
Grant user1 the read privilege on table st1:
login(`admin1, "123456")
grant("user1", TABLE_READ, "st1")
After the grant
operation is performed, access control is automatically added. Now user1 is granted the read privilege but not write privilege:
login("user1","123456")
select * from st1
insert into st1 values(1970.01.01T00:00:00.001,`sym,1)
// Error: No access to table [st1]
user1's privilege can be revoked:
login(`admin1, "123456")
revoke("user1", TABLE_READ, "st1")
login("user1","123456")
select * from st1
// Error: No access to shared table [st1]
Use addAccessControl
to restrict access to a shared in-memory table to the table creator and administrators.
share table(1:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as st2
addAccessControl(st2)
6.3 Streaming Data
6.3.1 Streaming Engines
Users can call addAccessControl
to restrict other users from accessing the streaming engines.
Examples:
Log in as user1 to create a streaming engine "engine1":
login(`admin, `123456)
createUser("user1", "123456");
createUser("user2", "123456");
login("user1", "123456")
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as trades
output1 = table(10000:0, `time`sym`sumVolume, [TIMESTAMP, SYMBOL, INT])
engine1 = createTimeSeriesEngine(name="engine1", windowSize=600, step=600, metrics=<[sum(volume)]>, dummyTable=trades, outputTable=output1, timeColumn=`time, useSystemTime=false, keyColumn=`sym, garbageSize=50, useWindowStartTime=false)
Add access control to engine1:
addAccessControl(engine1)
user2 fails to ingest data to the engine:
login("user2", "123456")
insert into engine1 values(2018.10.08T01:01:01.785,`A,10)
// ERROR: No access to table [engine1]
Grant user2 the write privilege on the engine:
login(`admin, `123456) grant("user2", TABLE_WRITE, "engine1")
user2 is allowed to ingest data to the engine:
login("user2", "123456")
insert into engine1 values(2018.10.08T01:01:01.785+1..600,take(`A,600),1..600)
6.3.2 Subscription From Stream Tables to Stream Tables
Before subscribing to a stream table, a user must have the TABLE_READ privilege to read the publishing table. Both TABLE_READ and TABLE_WRITE privileges to the stream table where the subscribed data will be saved are required.
Example 1. user1 creates stream tables pubTable and output1, and calls addAccessControl
.
login(`admin, `123456)
createUser(`user1, "123456");
login(`user1, "123456")
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as pubTable
share streamTable(10000:0, `time`sym`sumVolume, [TIMESTAMP, SYMBOL, INT]) as output1
addAccessControl(`pubTable)
addAccessControl(`output1)
Grant user2 TABLE_READ on table pubTable, and TABLE_READ and TABLE_WRITE on the table output1.
login(`admin, `123456)
grant("user2", TABLE_READ, "pubTable")
grant("user2", TABLE_WRITE, "output1")
grant("user2", TABLE_READ, "output1")
login(`user2, "123456")
subscribeTable(tableName="pubTable", actionName="engine1", offset=0, handler=append!{output1}, msgAsTable=true);
Example 2. When the publisher and subscriber are not on the same node, the applied object must be specified in the "nodeAlias:tableName" format.
Create pubTable on a remote NODE2 and add access control:
login(`admin, `123456)
def pubTable(){
share (streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]), `pubTable)
addAccessControl(`pubTable)
}
rpc(`NODE2,pubTable)
share streamTable(10000:0, `time`sym`sumVolume, [TIMESTAMP, SYMBOL, INT]) as output1
A local subscriber must have the TABLE_READ access privilege to pubTable on NODE2:
createUser(`user1, "123456");
grant("user1", TABLE_READ, "NODE2:pubTable")
login(`user1, "123456")
subscribeTable(server=`NODE2,tableName="pubTable", actionName="engine1", offset=0, handler=append!{output1}, msgAsTable=true);
6.3.3 Subscription From Stream Tables to DFS Tables
Before subscribing to a stream table, a user must have the TABLE_READ privilege to read the publishing table. The TABLE_WRITE privilege to the DFS table where the subscribed data will be saved is also required.
Examples:
Log in as admin to create a stream table "pubTable" and a DFS table "dfs://valuedb/pt":
login(`admin, `123456)
dbName = "dfs://valuedb"
t = table(10000:0, `time`sym`sumVolume, [TIMESTAMP, SYMBOL, INT])
insert into t values(2018.10.08T01:01:01.785+1..600,take(`A,600),1..600)
if(existsDatabase(dbName)){
dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt= db.createPartitionedTable(t, "pt", "sumVolume").append!(t)
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as pubTable
Grant user2 the TABLE_WRITE privilege on the DFS table:
createUser(`user2, "123456");
grant("user2", TABLE_WRITE,"dfs://valuedb/pt")
user2 can subscribe to the table:
login(`user2, "123456")
def savepubTableToDFS(mutable dfspubTable, msg): dfspubTable.append!(msg)
subscribeTable(tableName="pubTable", actionName="agg1", offset=0, handler=savepubTableToDFS{pt}, msgAsTable=true);
login(`admin, `123456)
select * from loadTable(dbName,`pt)
6.4 Function Views
Function views provide a flexible way to manage user access on databases and tables.
- Administrators can add or delete any function views. Users with VIEW_OWNER permission can add function views and delete the views they have created.
- Users with VIEW_EXEC permission can get the result of a function view even if they don't have access to the involved data.
- If a function view has the same name as a local function, the system will parse the function name as a local function.
- Some DDL and DML operations can be performed with only VIEW_EXEC privilege when included in function views.
- VIEW_EXEC can be applied to global (*), module level (<namespace>::*), or function view level. If functions of a module are added as views using
addFunctionName(moduleName)
, the fully-qualified namespace can be used for specifying a module-level access control.
Example 1. Define a function view:
def countTradeAll(){
return exec count(*) from loadTable("dfs://TAQ","Trades")
}
addFunctionView(countTradeAll)
grant("user1",VIEW_EXEC,"countTradeAll")
Log in as user1 to execute the function view:
countTradeAll()
In this example, though user1 does not have the privilege to access table "dfs://TAQ/Trades", user1 can still get the number of rows in the table with countTradeAll()
.
Example 2. Function views can take parameters. This example creates a function view to get the trades data of a specified stock on a specified date:
def getTrades(s, d){
return select * from loadTable("dfs://TAQ","Trades") where sym=s, date=d
}
addFunctionView(getTrades)
grant("user1",VIEW_EXEC,"getTrades")
Similarly, though user1 does not have the privilege to access table "dfs://TAQ/Trades", user1 can execute getTrades
with specified stock symbol and date:
getTrades("IBM", 2018.07.09)
Example 3.
Declare a module myModule and define functions f1
and f2
within the module.
module dir1::myModule
def f1(){
print "dir1::myModule::f1"
}
def f2(){
print "dir1::myModule::f2"
}
After referencing the module with use
, all functions within the module are added as function views.
use dir1::myModule
addFunctionView(moduleName="dir1::myModule")
getFunctionViews()
name | body |
---|---|
dir1::myModule::f1 | def f1(){ print("dir1::myModule::f1")} |
dir1::myModule::f2 | def f2(){ print("dir1::myModule::f2")} |
Grant VIEW_EXEC privilege on all function views within dir1::myModule
to user1:
grant(userId="user1", accessType=VIEW_EXEC, objs="dir1::myModule::*")
Deny VIEW_EXEC privilege on all function views within dir1::myModule
to user2:
deny(userId="user2", accessType=VIEW_EXEC, objs="dir1::myModule::*")
Note that the namespace must be specified to the module level. An error will be raised if the access control is set on dir1
.
grant(userId="user1", accessType=VIEW_EXEC, objs="dir1::*")
// Error:The Module [dir1::*] does not exist.
6.5 Scheduled Jobs
Scheduled jobs are automated tasks to execute on a specific time or recurring basis.
- A user who creates a scheduled job does not require the access privileges on the involved objects. However, if the user does not have privileges to access the objects when the scheduled job is set to be executed, the scheduled job will not be executed.
- With command
deleteScheduledJob
, administrators can delete all scheduled jobs, while users can only delete scheduled jobs created by themselves.
login("user1","123456")
def readTable(){
read_t1=loadTable("dfs://db1","t1")
return exec count(*) from read_t1
}
scheduleJob("readTableJob","read DFS table",readTable,minute(now()),date(now()),date(now())+1,'D');
In the example above, job readTable
can be scheduled even if user1 does not have privilege to read table "dfs://db1/t1". When the job is set to be executed, however, if "user1" still does not have privilege to read table "dfs://db1/t1", the job will not be executed.
6.6 Execution of Scripts
The scope of SCRIPT_EXEC and TEST_EXEC is global.
Executing function run
, or executing scripts on clients requires SCRIPT_EXEC privilege:
login(`admin, `123456)
createUser(`user1, "123456");
grant(`user1,SCRIPT_EXEC)
login(`user1, "123456")
run("test.txt")
Executing function test
, or executing unit tests on clients requires SCRIPT_EXEC privilege:
login(`admin, `123456)
createUser(`user1, "123456");
grant(`user1,TEST_EXEC)
login(`user1, "123456")
test("test.txt")
6.7 Memory Usage
You can call grant
and specify accessType = QUERY_RESULT_MEM_LIMIT to set the memory limit for the result of each query for a user, or accessType = TASK_GROUP_MEM_LIMIT to set the memory limit of a task group sent from the current node. It has similar functions with functions setMemLimitOfQueryResult
and setMemLimitOfTaskGroupResult
, but grant
allows you to set the limit for a specific user (currently a group is not supported).
Set the memory limit of a query at 4 GB for user1:
grant("user1", QUERY_RESULT_MEM_LIMIT, 4)
Set the memory limit of a task group at 4 GB for user1:
grant("user1", TASK_GROUP_MEM_LIMIT, 4)
7. Secure Communication with HTTPS
DolphinDB supports HTTPS for secure communication over the web.
7.1 Enable HTTPS
There are 2 ways to enable HTTPS:
- Add
enableHTTPS=true
in the configuration file of the cluster controller (controller.cfg). - Add
-enableHTTPS true
in the command line when starting the cluster controller.
./dolphindb -enableHTTPS true -home master -publicName www.psui.com -mode controller -localSite 192.168.1.30:8500:rh8500 -logFile ./log/master.log
7.2 HTTPS Certificate
You need to install server authentication certificate at each server in DolphinDB for secure connections. There should be a certificate on the controller and each of the agent nodes. Data nodes use the certificate located at the agent node on the same server.
7.2.1 Certificate Authority (CA) Certificate
Get a certificate from a certificate authority, rename it as server.crt, and copy to the folder keys under the home directory of the controller. If the folder keys does not exist, we need to create it. The certificate does not need to be installed as it is from a certificate authority. This is recommended in most use cases.
7.2.2 Self-Signed Authentication Certificate
Sometimes we may want to install a self-signed authentication certificate. For example, in a small isolated cluster or in a research (non-production) setting. Please take the following steps:
(1) Set the configuration parameter publicName as the domain name of the computer.
Add publicName=www.ABCD.com
to the configuration file (controller.cfg) for the controller in a cluster, or add -publicName www.ABCD.com
when starting the controller in the command line.
(2) Check if the authentication certificate and private key have been generated.
Start the controller node and check if the certificate file server.crt and the private key for the server serverPrivate.key exist in the folder keys under the home directory.
(3) Install the self-signed certificate to the certificate authority of the web browser.
In Google Chrome, choose Settings -> Advanced -> Manage certificates -> AUTHORITIES-> Import to install the self-signed certificate server.crt. In other web browsers, there might be slight differences in this process.
Now enter https://www.abcd.com:8500
in the web browser to connect to DolphinDB cluster manager, where 8500 is the port number of the controller. If you see a green lock sign and "Secure" in the address bar, the authentication certificate has been successfully installed.
8. SSO (Single Sign-On)
On the web-based cluster manager, we can click any data node to open a notebook on this node. The data node may be located on a different physical server from the controller node. DolphinDB provides SSO (Single Sign-On) so that users don't need to log in the system again when visiting different servers in a cluster.
DolphinDB provides 2 API functions for SSO:
getAuthenticatedUserTicket()
: issue the current user's encrypted ticketauthenticateByTicket(ticket)
: use the ticket generated bygetAuthenticatedUserTicket()
to log in the system.
DolphinDB developers can use these API functions to develop more functionalities.