I am new to survival analysis. I tried to using CoxPHFitter, But I came across this error. numpy.linalg.linalg.LinAlgError: Matrix is singular.
After went through this error, I came to know one of my column has non invertible matrix.
So what should I do now? Can't I use that column? If So, What is the conclusion I can come up with that column?
Full stack trace:
Traceback (most recent call last): File "surv_model.py", line 79, in cph.fit(X, 'T', event_col='label') File "/usr/local/lib/python2.7/dist-packages/lifelines/fitters/coxph_fitter.py", line 165, in fit step_size=step_size) File "/usr/local/lib/python2.7/dist-packages/lifelines/fitters/coxph_fitter.py", line 253, in _newton_rhaphson inv_h_dot_g_T = spsolve(-h, g.T, sym_pos=True) File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/basic.py", line 251, in solve _solve_check(n, info) File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/basic.py", line 31, in _solve_check raise LinAlgError('Matrix is singular.') numpy.linalg.linalg.LinAlgError: Matrix is singular.
I'm using python lifelines
You can try using the Moore-Penrose inverse of a matrix, which always exists. But be aware that in case of non-invertible matrices, this is only a least-squares fit to the optimal solution.
Re-thinking your problem, the comments are correct: Add a regularization parameter. This actually seems to be a known problem: https://github.com/sebp/scikit-survival/issues/28#issuecomment-370918386