kdb+

extracting info from a table column which is a list of lists in kdb


I have the following linear regression functions

// linear regression y vs x
// returns: intercept slope
linear:{
    if[not (count x)=(count y); show "x and y must have the same length"; exit 1];
    fit:{(enlist x) lsq y xexp/:til 1+z};
    fit[x; y; 1]
 };

// takes X and Y as a list of lists
linearLL:{[X;Y] t:([] x:X; y:Y); raze {9h$linear[x`x; x`y]} each t};

\d .

and I would like to get the intercept and slope from lr

/ sliding window from https://code.kx.com/q/kb/programming-idioms/
swin:{[f;w;s] f each {1_x,y}\[w#0; s]}

p:1 2 3 0N 0N 0N 4 5 0N 6 0N 0N 0N 7 8 9;
p:1f*p;
time:2024.01.01+til count p;
n:4;
t:([] p: p; time: time);

t:update c:til count t from t;
t:select from t where p>0;
t:update x:swin[::;n;c] from t;
t:update y:swin[::;n;p] from t;
/t:update y:swin[::;n;p] from t;
t:update x:1f*({distinct x} each x) from t;
t:update y:({x except 0} each y) from t;
t:update y:({x except 0N} each y) from t;
t:delete c from t;
show t:update lr:.regression.linearLL[x;y] from t;

which gives (as expected):

p time       x           y        lr      
------------------------------------------
1 2024.01.01 ,0f         ,1f              
2 2024.01.02 0 1f        1 2f     -1   1  
3 2024.01.03 0 1 2f      1 2 3f   -1   1  
4 2024.01.07 0 1 2 6f    1 2 3 4f -2.5 1.9
5 2024.01.08 1 2 6 7f    2 3 4 5f -3.7 2.2
6 2024.01.10 2 6 7 9f    3 4 5 6f -3.9 2.2
7 2024.01.14 6 7 9 13f   4 5 6 7f -3.9 2.3
8 2024.01.15 7 9 13 14f  5 6 7 8f -5.5 2.5
9 2024.01.16 9 13 14 15f 6 7 8 9f -1.5 1.9

However I get a 'length error on invoking:

show t:update intercept:lr[0], slope:lr[1] from t;

I'm presuming the elements of lr are a list of a pair of elements, but somehow that's not the case. How can i extract the intercept and slope information?


Solution

  • q)update intercept:lr[;0],slope:lr[;1]from([]lr:(-1 1;-1 1;-2.5 1.9))
    lr       intercept slope
    ------------------------
    -1   1   -1        1
    -1   1   -1        1
    -2.5 1.9 -2.5      1.9
    

    lr[0] and lr[1] are the first and second elements of lr as opposed to the first and second elements of each element in lr.

    q)lr:exec lr from([]lr:(-1 1;-1 1;-2.5 1.9))
    q)lr[0]
    -1 1
    q)lr[2]
    -2.5 1.9
    q)lr[;0]
    -1
    -1
    -2.5
    q)lr[;1]
    1
    1
    1.9