My question is, if I need to perform an affine transformation that involve multiplying (rotation, scaling, shearing in y axis, shearing in x axis and translation) to achieve the following formula:

Does the following code is a right implementation of the above formula?

rotation=[cos(theta) sin(theta) 0; -sin(theta) cos(theta) 0; 0 0 1];
scaling=[lamdax 0 0; 0 lamda2 0; 0 0 1];
shearingY=[1 0 0; alphay 1 0; 0 0 1];
shearingX=[1 alphax 0; 0 1 0; 0 0 1];
translation=[1 0 0; 0 1 0; dx dy 1];
T=rotation*scaling*shearingY*shearingX*translation;
T = maketform('affine',T);
I2_hat=imtransform(I2,T);

Thank you very much in advance

Yes it does. You can leave the composition of the transformations as is. As a final note, you must make the translation the last step. The rotations, shearing and other operations assume that this is done at the origin. As such, do all of the operations you want, then move the transformed image after using your translation matrix. In MATLAB, because the operations are performed using pre-multiplication as opposed to post-multiplication the chain of transformations should appear from left to right. This is required if you are going to use imtransform. In other platforms, the chain of transforms should appear from right to left. In this mindset, the first transformation starts from the right, the second appears to the left of it and so on. Specifically, in MATLAB if you had N transformations, the final transform matrix should be:

T = T1 * T2 * ... * TN;

In other platforms, it would be:

T = TN * ... * T2 * T1;

You need to make sure that the last transform TN is the translation transform. If you translated first (i.e. made T1 the translation transform), all of the other transformations assume that you are performing these with respect to the origin, and that origin has moved because of the translation. Therefore, the points after translation will be assumed to be with respect to the origin (0,0) and not where (dx,dy) is, and so the rest of the operations will be incorrect.

You are performing this here fine and so you can leave it. The rest of your transformations you don't need to move them anywhere. These can be in any order because those are all independent transformations. You must leave the translation until the end.

Minor Note

maketform and imtransform are currently deprecated. If at all possible, use affine2d in place of maketform and imwarp in place of imtransform.