exceldatetimetimetime-seriespressure

How can I align high frequency time series data in microsoft excel


I have a barologger pressure time series that takes a reading every 15 minutes and a CT2X pressure time series that takes a measurement every 5 minutes. Unfortunately the time series is shifted and they don't align correctly. How can I filter my CT2X time series data so that it matches the time series of the barologger? How could I take an average of the three closest CT2X readings around the barologger time?

I have listed a picture "sample series columns" that ultimately shows what I wish to obtain.

I've tried the index-match tool in excel but I don't think I've used it correctly or if it can be done with this data.

Time-seriesData


Solution

  • Since you did not show averages as a part of what you expect for results, I chose to return a single closest value by aligning the values by

    I added a line and some information to the data you provided:
    enter image description here

    If you have 365, this can be done with a complex formula (you will need to use the Microsoft provided Excel Labs add-in to see how the various steps are evaluated):

    =LET(
        d, Pressures,
        bdt, CHOOSECOLS(d, 1),
        cdt, CHOOSECOLS(d, 3),
        minDif, BYROW(bdt, LAMBDA(arr, MIN(ABS(arr - cdt)))),
        allDif, ABS(bdt - TRANSPOSE(cdt)),
        pos, BYROW(HSTACK(minDif, allDif), LAMBDA(arr, XMATCH(INDEX(arr, 1), DROP(arr, , 1)))),
        closest, HSTACK(TAKE(d, , 2), CHOOSEROWS(DROP(d, , 2), pos)),
        f, FILTER(closest, minDif < TIME(0, 2, 0)),
        f
    )
    

    Or, this can also be accomplished using Power Query, available in Windows Excel 2010+ and Excel 365 (Windows or Mac)

    To use Power Query

    M-Code

    let
        Source = Excel.CurrentWorkbook(){[Name="Pressures"]}[Content],
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"Barologger Date/Time", type datetime}, {"Barologger LEVEL", type number}, {"CT2X Date/Time", type datetime}, {"CT2X Pressure", type number}}),
    
    #"Added Custom" =Table.AddColumn(#"Changed Type","Match CT2X", (r)=>
        let
    
        //generate a list of differences between each of the times
            difs = List.TransformMany(
                        #"Changed Type"[#"CT2X Date/Time"],
                        each {_ -r[#"Barologger Date/Time"]},
                        (a,b)=> Number.Abs(Number.From(b))),
    
        //determine the row numbers with the matching min times, using 2 minutes (2/1440) as the threshold
            pos = if List.Min(difs) < 2/1440
                    then List.PositionOf(difs,List.Min(difs))
                    else null
            in pos, Int64.Type),
    
            
        #"Filtered Rows" = Table.SelectRows(#"Added Custom", each [Match CT2X] <> null and [Match CT2X] <> ""),
        #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"CT2X Date/Time", "CT2X Pressure"}),
        #"Matching CT2X" = Table.TransformColumns(#"Removed Columns",{"Match CT2X", 
            each Table.SelectColumns(#"Added Custom",{"CT2X Date/Time", "CT2X Pressure"}){_},type [#"CT2X Date/Time"=datetime, CT2X Pressure=number]}),
        #"Expanded Match CT2X" = Table.ExpandRecordColumn(#"Matching CT2X", "Match CT2X", {"CT2X Date/Time", "CT2X Pressure"}, {"CT2X Date/Time", "CT2X Pressure"})
     
    in
        #"Expanded Match CT2X"
    

    Either method gives the following results from the data I posted:
    enter image description here