uproot

uproot: processing a TH2D using the uproot method .pandas()


I am very new to uproot and Python, but hopefully catching up quickly. I am wondering why the method .pandas() is creating such a weird table from a TH2D histogram:

myhisto = file["angular_distr_el/ID3_mol_e0_valid/EN_gate/check_cthetaEE_x"]
type(myhisto)

outputs:

uproot.rootio.TH2D

Finally, myhisto.pandas() returns:

        count   variance
cos(theta)  electron energy [eV]        
[-inf, -1.0)    [-inf, 10.0)    0.0 0.0
[10.0, 10.15)   0.0 0.0
[10.15, 10.3)   0.0 0.0
[10.3, 10.45)   0.0 0.0
[10.45, 10.6)   0.0 0.0
... ... ... ...
[1.0, inf)  [24.4, 24.549999999999997)  0.0 0.0
[24.549999999999997, 24.7)  0.0 0.0
[24.7, 24.85)   0.0 0.0
[24.85, 25.0)   0.0 0.0
[25.0, inf) 0.0 0.0
2244 rows × 2 columns

and myhisto.columns returns:

Index(['count', 'variance'], dtype='object')

Where can I find the documentation of the method .pandas() to understand what it is doing? Is there a way to reorganise myhisto in a DataFrame with the right columns?


Solution

  • After some fun but desperate browsing, I understand which type of object it is. It is a very clever way of creating sorted MultiIndex DataFrames. Just typing myhisto.index is possible to see it directly:

    MultiIndex([([-inf, -1.0),                [-inf, 10.0)),
                ([-inf, -1.0),               [10.0, 10.15)),
                ([-inf, -1.0),               [10.15, 10.3)),
                ([-inf, -1.0),               [10.3, 10.45)),
                ([-inf, -1.0),               [10.45, 10.6)),
                ([-inf, -1.0),               [10.6, 10.75)),
                ([-inf, -1.0),               [10.75, 10.9)),
                ([-inf, -1.0),               [10.9, 11.05)),
                ([-inf, -1.0),               [11.05, 11.2)),
                ([-inf, -1.0),               [11.2, 11.35)),
                ...
                (  [1.0, inf), [23.65, 23.799999999999997)),
                (  [1.0, inf), [23.799999999999997, 23.95)),
                (  [1.0, inf),               [23.95, 24.1)),
                (  [1.0, inf),               [24.1, 24.25)),
                (  [1.0, inf),               [24.25, 24.4)),
                (  [1.0, inf),  [24.4, 24.549999999999997)),
                (  [1.0, inf),  [24.549999999999997, 24.7)),
                (  [1.0, inf),               [24.7, 24.85)),
                (  [1.0, inf),               [24.85, 25.0)),
                (  [1.0, inf),                 [25.0, inf))],
               names=['cos(theta)', 'electron energy [eV]'], length=2244)
    

    The solution is to unstack or create a pivot table of the DataFrame. For this specific object, a pivot table is better, because of the presence of counts AND variance as columns in the original DataFrame. As an example:

    myhisto.unstack()
    
    count   ... variance
    electron energy [eV]    [-inf, 10.0)    [10.0, 10.15)   [10.15, 10.3)   [10.3, 10.45)   [10.45, 10.6)   [10.6, 10.75)   [10.75, 10.9)   [10.9, 11.05)   [11.05, 11.2)   [11.2, 11.35)   ... [23.65, 23.799999999999997) [23.799999999999997, 23.95) [23.95, 24.1)   [24.1, 24.25)   [24.25, 24.4)   [24.4, 24.549999999999997)  [24.549999999999997, 24.7)  [24.7, 24.85)   [24.85, 25.0)   [25.0, inf)
    cos(theta)                                                                                  
    [-inf, -1.0)    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
    [-1.0, -0.9)    0.0 1.0 1.0 0.0 0.0 2.0 0.0 2.0 0.0 1.0 ... 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0
    [-0.9, -0.8)    0.0 0.0 3.0 3.0 0.0 0.0 0.0 0.0 1.0 1.0 ... 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0
    [-0.8, -0.7)    0.0 0.0 1.0 2.0 0.0 1.0 1.0 2.0 1.0 1.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
    [-0.7, -0.6)    0.0 0.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 ... 1.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 0.0
    [-0.6, -0.5)    0.0 1.0 1.0 1.0 0.0 0.0 2.0 1.0 0.0 3.0 ... 0.0 1.0 0.0 1.0 1.0 
    
    **22 rows × 204 columns**
    

    vs.

    pivot_pipanda = pipanda.pivot_table(values="count", index="cos(theta)", columns="electron energy [eV]")
    
    electron energy [eV]    [-inf, 10.0)    [10.0, 10.15)   [10.15, 10.3)   [10.3, 10.45)   [10.45, 10.6)   [10.6, 10.75)   [10.75, 10.9)   [10.9, 11.05)   [11.05, 11.2)   [11.2, 11.35)   ... [23.65, 23.799999999999997) [23.799999999999997, 23.95) [23.95, 24.1)   [24.1, 24.25)   [24.25, 24.4)   [24.4, 24.549999999999997)  [24.549999999999997, 24.7)  [24.7, 24.85)   [24.85, 25.0)   [25.0, inf)
    cos(theta)                                                                                  
    [-inf, -1.0)    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
    [-1.0, -0.9)    0.0 1.0 1.0 0.0 0.0 2.0 0.0 2.0 0.0 1.0 ... 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0
    [-0.9, -0.8)    0.0 0.0 3.0 3.0 0.0 0.0 0.0 0.0 1.0 1.0 ... 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0
    [-0.8, -0.7)    0.0 0.0 1.0 2.0 0.0 1.0 1.0 2.0 1.0 1.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
    [-0.7, -0.6)    0.0 0.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 ... 1.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 0.0
    [-0.6, -0.5)    0.0 1.0 1.0 1.0 0.0 0.0 2.0 1.0 0.0 3.0 ... 0.0 1.0 0.0 1.0 1.0 0.0 1.0 0.0 0.0 0.0
    [-0.5, -0.3999999999999999) 0.0 0.0 2.0 0.0 1.0 1.0 3.0 2.0 3.0 1.0 ... 3.0 0.0 0.0 0.0 0.0 2.0 0.0 1.0 1.0 0.0
    

    and from here the standard methods of pandas are available!

    (To play with slicing techniques like loc[] and iloc[]: https://www.youtube.com/watch?v=tcRGa2soc-c)