DataFrame

• In lazy mode

  import pandas as pd
  
  df = pd.DataFrame({"A": [1,2,3], "B": ["a", "b", "C"]}, index=["A1", "A2", "A3"])
  df

  Output:

  A B

  A1 1 a

  A2 2 b

  A3 3 C

• In non-lazy mode

  df = pd.DataFrame({"A": [1,2,3], "B": ["a", "b", "C"]}, None, False)

  Output:

  A B

  0 1 a

  1 2 b

  2 3 C

  df = pd.DataFrame({"A": [1,2,3], "B": ["a", "b", "C"]}, [`x,`y,`z], False)

  Output:

  A B

  x 1 a

  y 2 b

  z 3 C

Create an in-memory table of DolphinDB, tableForDf.

timetag =[2018.01.02, 2018.01.03, 2018.01.04, 2018.01.08].toddb()
name = [`AAPL, `AAPL, `GS, `AAPL].toddb()
flag = [`A, `A, `B, `C].toddb()
p = [10, 20, 30, 40].toddb()
tableForDf = table(p as price, flag as bsFlag, name, timetag)

• In lazy mode

  When lazy = True, index can only be the name of a specified column in tableForDf.

  df = pd.DataFrame(tableForDf, 'timetag', True)
  df

  Output:

  timetag price bsFlag name

  2018.01.02 10 A AAPL

  2018.01.03 20 A AAPL

  2018.01.04 30 B GS

  2018.01.08 40 C AAPL

• In non-lazy mode

  When lazy = False, index cannot be the name of a specified column in tableForDf.

  level1 = ['0001', '0002', '0002', '0001']
  df = pd.DataFrame(tableForDf, level1, False)
  df

  Output:

  price bsFlag name timetag

  0001 10 A AAPL 2018.01.02

  0002 20 A AAPL 2018.01.03

  0002 30 B GS 2018.01.04

  0001 40 C AAPL 2018.01.08

Only lazy DataFrame can be created from a DFS table.

n=1000
month=take(seq(2000.01M, 2016.12M), n);
x=rand(1.0, n);
t=table(month, x);
dbName="dfs://test_pandas"
if(exists(dbName)):
dropDatabase(dbName)
db=database(dbName, ddb.VALUE, partitionScheme=seq(2000.01M, 2016.12M))
pt=db.createPartitionedTable(t, `pt, `month).append!(t)

// index must be the name of a specified column in the table and lazy must be True
df=pd.DataFrame(pt,index="month", lazy=True)  

df=pd.DataFrame(pt,lazy=False)   // If lazy is set to False, an error occurs: cannot create non-lazy DataFrame with segmented table

Create an in-memory table tableForDf.

timetag =[2018.01.02, 2018.01.03, 2018.01.04, 2018.01.08].toddb()
name = [`AAPL, `AAPL, `GS, `AAPL].toddb()
flag = [`A, `A, `B, `C].toddb()
p = [10, 20, 30, 40].toddb()
tableForDf = table(p as price, flag as bsFlag, name, timetag)

1. Accessing Data by Position

   A lazy DataFrame is a view of the original object and does not support accessing data by position. Take a non-lazy DataFrame df_nonlazy as an example.

   level = ['0001', '0002', '0002', '0001']
   df_nonlazy=pd.DataFrame(tableForDf, level, False)

   • Access an element

     df_nonlazy.iat[0, 3]

     Output: 2018.01.02

   • Access column(s)

     df_nonlazy.iloc[:,1]

     Output:

     0001 A

     0002 A

     0002 B

     0001 C

     dtype: STRING

     df_nonlazy.iloc[:,1:3] 

     Output:

     bsFlag name

     0001 A AAPL

     0002 A AAPL

     0002 B GS

     0001 C AAPL

     df_nonlazy.iloc[:,2:]

     Output:

     name timetag

     0001 AAPL 2018.01.02

     0002 AAPL 2018.01.03

     0002 GS 2018.01.04

     0001 AAPL 2018.01.08

     df_nonlazy.iloc[:,:3] # select col0~col3

     Output:

     price bsFlag name

     0001 10 A AAPL

     0002 20 A AAPL

     0002 30 B GS

     0001 40 C AAPL

   • Access row(s)

     df_nonlazy.iloc[1]

     Output:

     price 20

     bsFlag A

     name AAPL

     timetag 2018.01.03

     dtype: ANY

     df_nonlazy.iloc[1:3] 

     Output:

     price bsFlag name timetag

     0002 20 A AAPL 2018.01.03

     0002 30 B GS 2018.01.04

   • Access pair(s) of row(s) and column(s)

     df_nonlazy.iloc[[0,3], [2]]

     Output:

     name

     0001 AAPL

     0001 AAPL

     df_nonlazy.iloc[0:3, 0:1]

     Output:

     price

     0001 10

     0002 20

     0002 30

     df_nonlazy.iat[3, 3]

     Output:m 2018.01.08

2. Accessing Data by Label

   • Access data by column label(s)

     df_nonlazy.price // Accessing data using the DataFrame.columns method is not supported

     df_nonlazy['price']

     Output:

     0001 10

     0002 20

     0002 30

     0001 40

     dtype: INT

     df_nonlazy[['timetag', 'price']]

     Output:

     timetag price

     0001 2018.01.02 10

     0002 2018.01.03 20

     0002 2018.01.04 30

     0001 2018.01.08 40

     df_nonlazy.loc[:, ['timetag', 'price']]

     Output:

     timetag price

     0001 2018.01.02 10

     0002 2018.01.03 20

     0002 2018.01.04 30

     0001 2018.01.08 40

   • Access data by row label(s)

     df_nonlazy.loc[`0001]

     Output:

     price bsFlag name timetag

     0001 10 A AAPL 2018.01.02

     0001 40 C AAPL 2018.01.08

     df_nonlazy.loc[[`0001,`0002]]

     Output:

     price bsFlag name timetag

     0001 10 A AAPL 2018.01.02

     0002 20 A AAPL 2018.01.03

     0002 30 B GS 2018.01.04

     0001 40 C AAPL 2018.01.08

   • Access pair(s) of row(s) and column(s)

     df_nonlazy['name'][`0001]

     Output:

     0001 AAPL

     0001 AAPL

     dtype: STRING

     df_nonlazy.at[`0001, 'name']

     Output:

     0001 AAPL

     0001 AAPL

     dtype: STRING

     df_nonlazy.loc[[`0001, `0002], ['name']]

     Output:

     name

     0001 AAPL

     0002 AAPL

     0002 GS

     0001 AAPL

3. Accessing Data by Boolean Index

   df_nonlazy.loc[[True,True,False, False]]
   df_nonlazy.iloc[[True, True,False, False]]
   df_nonlazy[[True, True,False, False]]

   Output:

   price bsFlag name timetag

   0001 10 A AAPL 2018.01.02

   0002 20 A AAPL 2018.01.03

4. Accessing Data by Condition

   df_nonlazy[df_nonlazy["price"]>20]

   Output:

   price bsFlag name timetag

   0002 30 B GS 2018.01.04

   0001 40 C AAPL 2018.01.08

A lazy DataFrame does not support adding, changing, or removing data. Take a non-lazy DataFrame df_nonlazy as an example.

1. Adding data

   • Add a column to a DataFrame using the insert method

     df_nonlazy.insert(0, `tep, [33,33,37,37])

   • Add a column to a DataFrame by directly assigning values to the added column (currently not supported)

     row = {"price":30, "bsFlag":'B', "name":"GS", "timetag":2018.01.09}
     df_nonlazy.iloc[1] = row

2. Updating data

   // Create an example DataFrame
   data = {'A': [1, 2, 3], 'B': [4, 5, 6]}
   df = pd.DataFrame(data)
   
   // Use methods df.at and df.iat to change the value of a single element
   df.at[0, 'A'] = 10
   df.iat[1, 1] = 50
   
   // Use the method df.apply to apply a function to a column
   df['A'] = df['A'].apply(lambda x: x * 2)

   If a DataFrame is created from a table, the modification of its data through value assigning will also be applied to the original table.

   // Create an example DataFrame from an in-memory tablee
   t1=table([1,2,3].toddb() as value)
   df=pd.DataFrame(t1,lazy=False)
   
   // Use methods df.at and df.iat to change the value of a single element
   df.at[0, 'value']=10
   df.iat[1, 0] = 20
   t1

   Output:

   value

   10

   20

   3

3. Removing data

   • Remove elements using the drop method

     df_nonlazy.drop(['name'], axis=1)