javascriptmathraspberry-pimeannormalization

Ignore largely differing electronics input values


I have a physical, draggable device connected to a Raspberry Pi, which upon movement outputs its current position as keyboard input. The physical device can "have a position" in the range from 91 pixels to 955 pixels.

The pixels are supposed to control an image on a screen. The image should scroll up/down smoothly based on the handle's position/generated values. See below picture for visualization.

Visualization of setup

This works OK when dragging the handle downwards; the values are fairly sequential. But when dragging the handle it upwards, the electronics output random/jumpy values.

I need help with the maths to normalize the values and ignore large diffs in JavaScript.

I.e. the values might go smoothly from 92, 94, 96, 100, 105… for a while, and then suddenly jump to 847.

I can't fix the electronics unfortunately, so I need a programmatic solution.

Here's a real example of keyboard output received from the electronic device:

const values = [
  92,
  94,
  96,
  100,
  102,
  105,
  107,
  112,
  123,
  134,
  116,
  114, // Last acceptable position
  847, // Huge diff compared to previous value
  299,
  628, // Huge diff again
  559, // Not as big of a diff compared to the previous value, but too big compared to 114 which was the last acceptable value
  395,
  788,
  189,
  948,
  135, // ≈ 20 px difference to 114 would be OK
  907,
  190,
  853,
  134,
  981,
  34,
  940,
  64,
  899,
  214,
  168,
  174,
  182,
  187,
  189,
  191,
  193,
  195,
  198,
  200,
  204,
  209,
  211,
  214,
  217,
  220,
  222,
  225,
  231,
  229,
  232,
  225,
  233,
  230,
  245,
  241,
  237,
  241,
  239,
  242,
  57,
  275,
  125,
  242,
  238,
  245,
  240,
  243,
  230,
  244,
  233,
  360,
  241,
  248,
  240,
  249,
  251,
  260,
  32,
  1007,
  68,
  1018,
  243,
  711,
  498,
  440,
  765,
  199,
  951,
  121,
  932,
  148,
  274,
  271,
  214,
  271,
  285,
  267,
  351,
  246,
  376,
  102,
  412,
  61,
  580,
  0,
  336,
  270,
  391,
  269,
  279,
  282,
  284,
  288,
  291,
  293,
  298,
  305,
  309,
  311,
  313,
  318,
  322,
  324,
  330,
  332,
  337,
  332,
  339,
  980,
  120,
  835,
  370,
  557,
  667,
  273,
  909,
  77,
  761,
  324,
  873,
  104,
  1005,
  156,
  799,
  418,
  509,
  715,
  229,
  366,
  369,
  410,
  369,
  365,
  374,
  376,
  381,
  383,
  387,
  392,
  394,
  398,
  405,
  408,
  410,
  418,
  427,
  433,
  437,
  439,
  441,
  228,
  660,
  562,
  362,
  849,
  120,
  999,
  25,
  993,
  60,
  916,
  70,
  900,
  81,
  884,
  191,
  879,
  196,
  873,
  101,
  1007,
  168,
  781,
  444,
  479,
  749,
  198,
  961,
  101,
  959,
  103,
  956,
  40,
  946,
  48,
  934,
  56,
  808,
  268,
  918,
  68,
  904,
  78,
  886,
  460,
  472,
  480,
  489,
  497,
  499,
  501,
  509,
  520,
  526,
  528,
  530,
  536,
  544,
  551,
  565,
  558,
  556,
  558,
  532,
  558,
  556,
  584,
  500,
  558,
  551,
  540,
  612,
  559,
  579,
  581,
  583,
  581,
  590,
  596,
  598,
  600,
  606,
  615,
  622,
  626,
  628,
  632,
  646,
  659,
  962,
  687,
  699,
  705,
  710,
  714,
  720,
  729,
  740,
  757,
  753,
  775,
  748,
  757,
  204,
  775,
  789,
  802,
  811,
  813,
  815,
  818,
  827,
  830,
  832,
  836,
  841,
  845,
  847,
  850,
  852,
  842,
  852,
  828,
  851,
  849,
  852,
  849,
  852,
  850,
  852,
  850,
  853,
  851,
  864,
  829,
  852,
  850,
  852,
  850,
  852,
  854,
  852,
  861,
  326,
  871,
  129,
  880,
  887,
  894,
  899,
  903,
  905,
  908,
  910,
  912,
  915,
  919,
  921,
  927,
  932,
  934,
  939,
  944,
  947,
  949,
  954,
  978,
  923,
  1002,
  872,
  965,
  877,
  964,
  912,
  987,
  957,
  966,
  973,
  962,
  978,
  939,
  1021,
  723,
  989,
  899,
  1023,
  961,
  963,
  959,
  1020,
  957,
  962,
  960,
  968,
  944,
  980,
  961,
  963,
  961]

I tried calculating the difference between the previous position and the received value, and skipping it if it was too big (i.e. over 20 pixels). This works OK when dragging the handle downwards because the received input is within the limit, but not with the dataset received when going upwards.

The jump in values becomes too big at some point, and then the graphic becomes "stuck" because the handle has been dragged too rapidly and it's (seemingly) impossible to determine its new "real" location.

I.e. if looking at the above data set, 114 would have been the "last acceptable position" (index 11 in the array) and then the new acceptable position would be 135 (index 20) and then I suppose somewhere around 168 after that.

The handle is user-controlled (i.e. a human being dragging the handle), so there's no way to programmatically "stop" it or alter its position, or put any resistance on it.

I've also tried to use the mean value from the last 3 and 5 values, but that didn't seem like the best approach because sometimes there are too many incorrect consecutive values (10-ish), which distorts the last known "real" position. Or should I just increase the amount of values included in the mean? How then to handle "edge cases" like when the draggable handle is at the very top/bottom and there are no previous values to refer to?

Any help/ideas/creative approaches are much welcome!


Solution

  • Edit Mon, April 29, 2024: Ended up solving this as follows (and it works perfectly for the electronics in this case):

    1. Store the last keyboard input/value in an array, limit the length to 10.
    2. Copy the 10 values to a temporary array and sort it
    3. If there's more than 6 values, remove the first 3 elements and last 3 elements from the array to get the middle percentile.
    4. Calculate the average of the remaining values.
    5. If the average value differs less than 100 pixels, save this as the "last known OK position" and update the graphic's position to it.

    I also added if statements to cap the maximum and minimum values, just to be sure the graphic is never out of limits.


    Monday, April 29, 2024 sometime in the early morning: Ah, apparently this is called "outliers" and the common approach would be to use interquartile ranges (IQR) to get the middle 50 % of the values.

    Here's an example outliers filter in JavaScript: https://gist.github.com/ogun/f19dc055e6b84d57e8186cbc9eaa8e45

    Haven't gotten it quite to work yet for the smooth scrolling, but thought I'd add the terminology here as a reference if someone else stumbles across this and isn't a stats wiz.