roffsetzerodrift

Zero offset for sensor data set


I can't find the answer in search but maybe I'm using the wrong terms. I am trying to zero-offset depth sensor data which drifts.

I have a large dataset from months of data with 5 second increments, it looks like this:

  Depth Temperature Light_Level          Local_time
1  64.0       27.55         148 2013-10-14 12:30:00
2  65.5       27.50         148 2013-10-14 12:30:05
3  65.5       27.40         148 2013-10-14 12:30:10
4  66.0       27.35         148 2013-10-14 12:30:15
5  67.5       27.25         147 2013-10-14 12:30:20

I need to adjust so 0 is the minimum value for depth. In order to do this, I am using this code:

daily_depths <- with(dataset_1, aggregate(Depth ~ as.Date(Local_time), FUN =  function(x) c(Min = min(x), Max = max(x)))) #pulls out min daily depth for each day

min_max <- do.call(data.frame, daily_depths) #converts to a df with date, min and max values

MA5 <- stats::filter(min_max$Depth.Min, rep(1,5), sides = 2)/5 #calculates a moving average from 5 values centered on the current value

MA5 is output as a Time Series with Start = 1, End = 180 (or length of dataset in days), Frequency = 1

e.g. printed: [1] NA NA -0.6 -0.6 -0.6 -0.5 -0.5 -0.5 -0.5 -0.5 ....... -0.5 NA NA

Now I want to subtract the daily value from all data points of that day. There are a few issues:

  1. the first two and last two days don't have a 5 day moving average. I would like to create an average and replace the NA's with these numbers. For the 2nd-to-last and last that's a simple modification of the MA, using sides = 1
  2. for the first and second I'm not sure how given the sides function doesn't allow for using future values.
  3. Once I have those numbers, how do I then replace the NA with them in the MA5 time series

Lastly, I would need to add or subtract the daily moving average value from every depth value of the corresponding day, this will depend on whether the number is positive or negative, to correct back to 0. Any help with this would be great. Many thanks


Solution

  • I ended up creating a function in IGOR using a moving box window to calculate the lowest value of the box, then create a moving average across the data. Then using the wave I created I subtracted this wave from the data, shifting the whole dataset by the adjusted value. Finally I replace any values below 0 with 0's so that depth is always 0 or greater.