pandasdataframejuliapycall

Julia pandas - how to append dataframes together


Working with Julia 1.0 I have a large numbers of data frames which I read into Julia using pandas (read_csv) and I am looking for a way to append them all together into a single big data frame. For some reason the "append" function does not do the trick. A simplified example below:

using Pandas 

df = Pandas.DataFrame([[1, 2], [3, 4]], columns=['A', 'B'])

df2 = Pandas.DataFrame([[5, 6], [7, 8]], columns=['A', 'B'])

df[:append](df2)  #fails

df.append(df2)    #fails

df[:concat](df2)  #fails

vcat(df,df2)       

The last step works but produces a 2 element Array with each element being a DataFrame

Any ideas on how to stack the two dataframes one under the other?


Solution

  • This seems to work

    julia> df = Pandas.DataFrame([[1, 2], [3, 4]], columns=[:A, :B])
       A  B
    0  1  2
    1  3  4
    
    
    julia> df2 = Pandas.DataFrame([[5, 6], [7, 8]], columns=[:A, :B])
       A  B
    0  5  6
    1  7  8
    
    
    julia> df.pyo[:append](df2, ignore_index = true )
    PyObject    A  B
    0  1  2
    1  3  4
    2  5  6
    3  7  8
    

    Notes:

    For reference, here's the equivalent in julia DataFrames:

    julia> df  = DataFrames.DataFrame( [1:2, 3:4], [:A, :B]);
    julia> df2 = DataFrames.DataFrame( [5:6, 7:8], [:A, :B]);
    julia> append!(df, df2)
    4×2 DataFrames.DataFrame
    │ Row │ A │ B │
    ├─────┼───┼───┤
    │ 1   │ 1 │ 3 │
    │ 2   │ 2 │ 4 │
    │ 3   │ 5 │ 7 │
    │ 4   │ 6 │ 8 │