limitstatastockshft

Stata: how to get observation value 5 minutes ahead with gabbed time data


I got high frequency data from a limit order book in Stata. Time does not have a regular interval, and some observations are at the same time (in milliseconds). For each observation I need to get the midpoint 5 minutes later in a separate column. So for observation 1 the midpoint would be 10.49, because the last midpoint closest to 09:05:02.579 would be 10.49.

How to do this in Stata?

datetime                         midpoint
12/02/2012 09:00:02.579          10.5125
12/02/2012 09:00:03.471          10.5125
12/02/2012 09:00:03.471          10.5125
12/02/2012 09:00:03.471          10.51
12/02/2012 09:00:03.471          10.51
12/02/2012 09:00:03.549          10.505
12/02/2012 09:00:03.549          10.5075
   ......
12/02/2012 09:04:59.785          10.495
12/02/2012 09:05:00.829          10.4925
12/02/2012 09:05:01.209          10.49
12/02/2012 09:05:03.057          10.4875
12/02/2012 09:05:05.055          10.485
 .....

Solution

  • My approach would be

    1. generate a new data set shifted by five minutes
    2. append this shifter data set
    3. find closest before and after observations to your five minute delta
    4. use some criteria to pick the better of these two values

    You specified closest, but you might want to add some other criteria depending on your book. Also, you mentioned more than one value at a given ms tick, but without more information I'm not sure how to handle that. Do you want to combine those midpoints first? Or are they different stocks?

    Here's some code that implements the basics of the approach above.

    clear
    version 11.2
    set seed 2001
    
    * generate some data
    set obs 100000
    generate double dt = ///
        tc(02dec2012 09:00:00.000) + 1000*_n + int(100*rnormal())
    format dt %tcDDmonCCYY_HH:MM:SS.sss
    sort dt
    generate midpt = 100
    replace midpt = ///
        round(midpt[_n - 1] + 0.1*rnormal(), 0.005) if (_n != 1)
    
    * add back future midpts
    preserve
    tempfile future
    rename midpt fmidpt
    rename dt fdt
    generate double dt = fdt - tc(00:05:00.000)
    save `future'
    restore
    append using `future'
    
    * generate midpoints before and after 5 minutes in the future
    sort dt
    foreach v of varlist fdt fmidpt {
        clonevar `v'_b = `v'
        replace `v'_b = `v'_b[_n - 1] if missing(`v'_b)
    }
    
    gsort -dt
    foreach v of varlist fdt fmidpt {
        clonevar `v'_a = `v'
        replace `v'_a = `v'_a[_n - 1] if missing(`v'_a)
    }
    
    format fdt* %tcDDmonCCYY_HH:MM:SS.sss
    
    * use some algorithm to pick correct value
    sort dt    
    generate choose_b = ///
        ((dt + tc(00:05:00.000)) - fdt_b) < (fdt_a - (dt + tc(00:05:00.000))) 
    generate fdt_c = cond(choose_b, fdt_b, fdt_a)
    generate fmidpt_c = cond(choose_b, fmidpt_b, fmidpt_a)
    format fdt_c %tcDDmonCCYY_HH:MM:SS.sss