rglmnet

Error in y - predmat : non-numeric argument to binary operator


Trying to use R cv.glmnet() for cross validation on loans data. I have a data set on loan data (Kaggle) and have already split into train, test. Separated the y response from the predictive variables in select(1) and select(-1). Created matrix so as to avoid the "Error in storage.mode(y) <- "double" : 'list' object cannot be coerced to type 'double' " problem earlier.

Now seeking to run cv.glmnet() for cross validation, but this error stops me now.

"Error in y - predmat : non-numeric argument to binary operator"

Error in non-numeric argument, yet all my data is numeric, save for one factor for response y.

As a side question, what is the predmat in "y - predmat" refer to?

x_vars <- as.matrix(data.sample.train.split %>% select(-1)) 
y_resp <- as.matrix(data.sample.train.split %>% select(1))

cv_output <- cv.glmnet(x_vars, y_resp, type.measure = "deviance", nfolds = 5)
cv_output <- cv.glmnet(x_vars, y_resp, 
                       type.measure = "deviance",
                       lambda = NULL, 
                       nfolds = 5)

I am also considering to try this function:

ddd.lasso <- cv.glmnet(x_vars, y_resp, alpha = 1, family = "binomial")
ddd.model <- glmnet(x_vars, y_resp, alpha = 1, family = "binomial", lambda = ddd.lasso$lambda.min)

Data sample is as follows, just some of the columns:

c("loan_amnt", "funded_amnt", 
"funded_amnt_inv", "grade", "emp_length", "annual_inc", "dti", 
"mths_since_last_delinq", "mths_since_last_record", "open_acc", 
"pub_rec", "revol_bal", "revol_util", "total_acc", "out_prncp", 
"out_prncp_inv", "total_pymnt", "total_pymnt_inv", "total_rec_prncp", 
"total_rec_int", "total_rec_late_fee", "recoveries", "collection_recovery_fee", 
"last_pymnt_amnt", "collections_12_mths_ex_med", "acc_now_delinq"
)))
     loan_amnt funded_amnt funded_amnt_inv grade emp_length annual_inc   dti
3        10000       10000       10000.000    60         10   49200.00 20.00
10       10000       10000       10000.000    60          4   42000.00 18.60
14       20250       20250       19142.161    60          3   43370.00 26.53
17       15000       15000       15000.000    80          2   92000.00 29.44
18        4000        4000        4000.000    80         10  106000.00  5.63
31        4400        4400        4400.000    40         10   55000.00 20.01
35       10000       10000       10000.000   100         10   60000.00 12.74
37       25600       25600       25350.000    80          9  110000.00 15.71
41       10000       10000       10000.000    80          1   39000.00 18.58
64        9200        9200        9200.000    80          2   60000.00 19.96
72        7000        7000        7000.000    80          4   39120.00 21.01
74        3500        3500        3500.000   100         10   83000.00  2.31
77        9500        9500        9500.000   100          7   50000.00  8.18
89       10000       10000       10000.000   100          1   43000.00 25.26
98        7000        7000        7000.000    80          1   30000.00 15.80
112      21600       21600       20498.266    20          8   60000.00 16.74
117       7200        7200        7200.000    80          5   48000.00 17.43
118      12000       12000       11975.000    60          1   57000.00 10.86
125      10000       10000       10000.000   100          5   70000.00 16.78
126       8000        8000        8000.000    60          3   28000.00 12.60
128       6000        6000        6000.000    60         10   94800.00 24.53
138      35000       35000       35000.000    80          2  168000.00  3.17
144      14000       14000       14000.000   100         10   66000.00 11.15
149       3000        3000        3000.000    60          5   71000.00 21.84
152      12000       12000       11975.000    80          2   60000.00 15.50
153       6000        6000        6000.000   100          3   34000.00 14.51
155       7000        7000        7000.000    80          7   82000.00 12.00
166      24250       18100       18075.000    -1          7  120000.00 12.96
170       2500        2500        2500.000    80          7   29000.00 18.70
172       4225        4225        4225.000    80          5   55200.00 17.61
180       6000        6000        6000.000    60          5   50000.00 15.58
192       5000        5000        5000.000    80          5   38004.00 23.78
193       8000        8000        8000.000    80          3   31000.00 16.22
199      12000       12000       12000.000    80          4   40000.00 22.20
203       3200        3200        3200.000    80          9   61200.00  2.16
209       5000        5000        5000.000    80          2   70000.00 20.06
220      13250       13250       13250.000    40         10   52000.00 23.70
224      12000       12000       12000.000   100         10   68000.00  7.08
     mths_since_last_delinq mths_since_last_record open_acc pub_rec revol_bal revol_util
3                        35                     59       10       0      5598       21.0
10                       61                    114       14       0     24043       70.2
14                       18                    107        8       0     17813       85.6
17                       54                     79        8       0     13707       93.9
18                       18                     97       12       0      6110       37.7
31                       68                    119        7       0     25237       99.0
35                       37                     93       11       0     14019       19.5
37                       11                    118       11       0     26088       62.0
41                       58                     17        5       0     12874       72.7
64                       39                     95        8       0     23299       78.7
72                       26                     33        8       1      9414       52.4
74                       35                     59        6       0      3092       13.4
77                       46                    118        8       0     13422       60.5
89                       59                    105        8       0      8215       37.2
98                       68                    101        7       0     15455       47.6
112                      23                     26        6       0     13354       78.1
117                      24                     19        7       0     16450       80.2
118                      47                     87        7       0      9273       81.5
125                      32                     92        9       0     10770       69.0
126                      66                    112        8       0      6187       54.3
128                      10                    101       13       0     71890       95.9
138                      22                     97       16       0      1099        1.4
144                      26                    102        7       0     12095       35.4
149                      59                    103        4       0     15072       88.7
152                      46                     94        7       0     12168       85.7
153                      70                     81        9       0     13683       64.8
155                      79                     83        6       0     25334       71.6
166                      66                    118        7       0     31992       99.0
170                      63                     99        5       0      2668       66.7
172                      69                    104        6       0      4055       73.7
180                      49                     94        8       0      7361       83.6
192                       5                     85       12       0     10023       57.3
193                      28                     77       13       0      2751       34.4
199                      78                    109        9       0     16273       55.5
203                      79                    113        5       1      2795       33.3
209                      27                     62       14       0     13543       54.2
220                      70                     86        8       0     15002       91.5
224                      21                     70        7       0     15433       55.6
     total_acc out_prncp out_prncp_inv total_pymnt total_pymnt_inv total_rec_prncp
3           37         0             0   12226.302        12226.30        10000.00
10          28         0             0   12519.260        12519.26        10000.00
14          22         0             0   27663.043        25417.68        20250.00
17          31         0             0   15823.480        15823.48        15000.00
18          44         0             0    4484.790         4484.79         4000.00
31          11         0             0    5626.893         5626.89         4400.00
35          18         0             0   10282.670        10282.67        10000.00
37          27         0             0   29695.623        29405.63        25600.00
41          10         0             0   11474.760        11474.76        10000.00
64          19         0             0   10480.840        10480.84         9200.00
72          26         0             0    7932.300         7932.30         7000.00
74          28         0             0    3834.661         3834.66         3500.00
77          13         0             0   10493.710        10493.71         9500.00
89          16         0             0   11264.010        11264.01        10000.00
98          11         0             0    8452.257         8452.26         7000.00
112         21         0             0   27580.750        24853.63        21600.00
117         10         0             0    8677.156         8677.16         7200.00
118         11         0             0   14396.580        14366.62        12000.00
125         18         0             0   10902.910        10902.91        10000.00
126         11         0             0    8636.820         8636.82         8000.00
128         30         0             0    7215.050         7215.05         6000.00
138         22         0             0   38059.760        38059.76        35000.00
144         46         0             0   15450.084        15450.08        14000.00
149         14         0             0    3723.936         3723.94         3000.00
152         21         0             0   13919.414        13890.44        12000.00
153         16         0             0    6857.261         6857.26         6000.00
155         31         0             0    8290.730         8290.73         7000.00
166         20         0             0   22188.250        22157.63        18100.00
170         13         0             0    2894.740         2894.74         2500.00
172         12         0             0    5081.023         5081.02         4225.00
180         14         0             0    7325.299         7325.30         6000.00
192         17         0             0    6534.430         6534.43         5000.00
193         29         0             0    8306.470         8306.47         8000.00
199         23         0             0   14006.680        14006.68        12000.00
203         17         0             0    3709.193         3709.19         3200.00
209         26         0             0    5501.160         5501.16         5000.00
220         18         0             0   15650.390        15650.39        13250.00
224         34         0             0   12554.010        12554.01        12000.00
     total_rec_int total_rec_late_fee recoveries collection_recovery_fee last_pymnt_amnt
3          2209.33           16.97000          0                       0          357.48
10         2519.26            0.00000          0                       0          370.46
14         7413.04            0.00000          0                       0         6024.09
17          823.48            0.00000          0                       0         2447.05
18          484.79            0.00000          0                       0         2638.77
31         1226.89            0.00000          0                       0          162.44
35          282.67            0.00000          0                       0         8762.05
37         4095.62            0.00000          0                       0          838.27
41         1474.76            0.00000          0                       0         5803.94
64         1280.84            0.00000          0                       0          365.48
72          932.30            0.00000          0                       0         4235.03
74          334.66            0.00000          0                       0          107.86
77          993.71            0.00000          0                       0         5378.43
89         1264.01            0.00000          0                       0            4.84
98         1452.26            0.00000          0                       0          238.06
112        5980.75            0.00000          0                       0        17416.49
117        1462.16           15.00000          0                       0           19.26
118        2396.58            0.00000          0                       0         5359.38
125         902.91            0.00000          0                       0         4152.52
126         636.82            0.00000          0                       0         6983.56
128        1215.05            0.00000          0                       0         1960.88
138        3059.76            0.00000          0                       0          272.59
144        1450.08            0.00000          0                       0         2133.17
149         723.94            0.00000          0                       0          107.29
152        1919.41            0.00000          0                       0          395.05
153         857.26            0.00000          0                       0          198.16
155        1290.73            0.00000          0                       0         2454.29
166        4088.25            0.00000          0                       0        16499.75
170         394.74            0.00000          0                       0         1168.50
172         856.02            0.00000          0                       0          146.48
180        1325.30            0.00000          0                       0          215.51
192        1534.43            0.00000          0                       0         1561.93
193         306.47            0.00000          0                       0         7778.22
199        2006.68            0.00000          0                       0         5971.51
203         509.19            0.00000          0                       0          317.41
209         501.16            0.00000          0                       0         3833.62
220        2400.39            0.00000          0                       0         9026.78
224         554.01            0.00000          0                       0          473.95
     collections_12_mths_ex_med acc_now_delinq
3                             0              0
10                            0              0
14                            0              0
17                            0              0
18                            0              0
31                            0              0
35                            0              0
37                            0              0
41                            0              0
64                            0              0
72                            0              0
74                            0              0
77                            0              0
89                            0              0
98                            0              0
112                           0              0
117                           0              0
118                           0              0
125                           0              0
126                           0              0
128                           0              0
138                           0              0
144                           0              0
149                           0              0
152                           0              0
153                           0              0
155                           0              0
166                           0              0
170                           0              0
172                           0              0
180                           0              0
192                           0              0
193                           0              0
199                           0              0
203                           0              0
209                           0              0
220                           0              0
224                           0              0

Solution

  • Looks like a incorrect glmnet family, I accidently chose the default 'deviance' for cv.glmnet, when in fact my data was binomial. My next solution is to figure out "Convergence for 1th lambda value not reached after maxit=100000 iterations; solutions for larger lambdas returned"

    Code that improved the solution:

    cv.lasso <- cv.glmnet(x_vars, y_resp, alpha = 1, family = "binomial", nfolds = 5)
    cv.model <- glmnet(x_vars, y_resp, alpha = 1, relax=TRUE, family="binomial", lambda=cv.lasso$lambda.min)