matlabmachine-learninghyperparametersgaussian-process

Squared covariance function of Gaussian process


This is my first attempt to write the covariance function. I have following values,

x = [-1.50 -1.0 -.75 -.40 -.25 0.00]; 
sf = 1.27;
ell = 1;
sn = 0.3;

The formula for squared exponential covariance function is enter image description here

The matlab code for that I have written as :

K = sf^2*exp(-0.5*(squareform(pdist(x)).^2)/ell^2)+(sn)^2*eye(Ntr,Ntr);

where sf is the signal standard deviation, ell the characteristic length scale, sn the noise standard deviation and Ntr the length of the training input data x. But this giving me no result. Is there is any mistake in my coding ?

And once I calculate, I want to summarize into matrix form as shown below , enter image description here

if x_ = 0.2 then how we can calculate :

a) K_ =[k(x_,x1) k(x_,x2)..........k(x_,xn)] and

b) K__ = k(x_,x_)

using matlab?


Solution

  • I obtain the following error message:

    Error using  + 
    Matrix dimensions must agree.
    
    Error in untitled3 (line 7)
    K = sf^2*exp(-0.5*(squareform(pdist(x)).^2)/ell^2)+(sn)^2*eye(Ntr,Ntr);
    

    indicating that your matrix dimensions do not agree. If you evaluate the parts of your code separately, you will notice that pdist(x) returns a 1x0 vector. In the documentation of pdist the expected format of x is explained:

    Rows of X correspond to observations, and columns correspond to variables

    So, instead of calculating pdist(x), you should calculate it for the transpose of x, i.e. pdist(x.'):

    K = sf^2*exp(-0.5*(squareform(pdist(x.')).^2)/ell^2)+(sn)^2*eye(Ntr,Ntr);
    

    As a conclusion, always read the error messages and documentation carefully, especially the expected format of your input arguments.

    Subquestions

    To calculate K for a specific value of x_ (x' in your mentioned formula), you can convert your given formula almost literally to MATLAB:

    K_ = sf^2*exp(-0.5*(x-x_).^2/ell^2)+(sn)^2*(x == x_);
    

    To calculate K__, you can use the formula above and setting x = x_, or you can simplify your formula to:

    K__ = sf^2+sn^2;