[SOLVED] Loss is increasing from first epoch itself

Loss is increasing from first epoch itself

I am training my siamese network for nlp. I have used lstm in it. and BCELoss. My loss is increasing from the first epoch. The first 36 epoch loss is

error after  0 is 
 272.4357
[torch.FloatTensor of size 1]

error after  1 is 
 271.8972
[torch.FloatTensor of size 1]

error after  2 is 
 271.5598
[torch.FloatTensor of size 1]

error after  3 is 
 271.6979
[torch.FloatTensor of size 1]

error after  4 is 
 271.7315
[torch.FloatTensor of size 1]

error after  5 is 
 272.3965
[torch.FloatTensor of size 1]

error after  6 is 
 273.3982
[torch.FloatTensor of size 1]

error after  7 is 
 275.1197
[torch.FloatTensor of size 1]

error after  8 is 
 275.8228
[torch.FloatTensor of size 1]

error after  9 is 
 278.3311
[torch.FloatTensor of size 1]

error after  10 is 
 277.1054
[torch.FloatTensor of size 1]

error after  11 is 
 277.8418
[torch.FloatTensor of size 1]

error after  12 is 
 279.0189
[torch.FloatTensor of size 1]

error after  13 is 
 278.4090
[torch.FloatTensor of size 1]

error after  14 is 
 281.8813
[torch.FloatTensor of size 1]

error after  15 is 
 283.4077
[torch.FloatTensor of size 1]

error after  16 is 
 286.3093
[torch.FloatTensor of size 1]

error after  17 is 
 287.6292
[torch.FloatTensor of size 1]

error after  18 is 
 297.2318
[torch.FloatTensor of size 1]

error after  19 is 
 307.4176
[torch.FloatTensor of size 1]

error after  20 is 
 304.6649
[torch.FloatTensor of size 1]

error after  21 is 
 328.9772
[torch.FloatTensor of size 1]

error after  22 is 
 300.0669
[torch.FloatTensor of size 1]

error after  23 is 
 292.3902
[torch.FloatTensor of size 1]

error after  24 is 
 300.8633
[torch.FloatTensor of size 1]

error after  25 is 
 305.1822
[torch.FloatTensor of size 1]

error after  26 is 
 333.9984
[torch.FloatTensor of size 1]

error after  27 is 
 346.2062
[torch.FloatTensor of size 1]

error after  28 is 
 354.6148
[torch.FloatTensor of size 1]

error after  29 is 
 341.3568
[torch.FloatTensor of size 1]

error after  30 is 
 369.7580
[torch.FloatTensor of size 1]

error after  31 is 
 366.1615
[torch.FloatTensor of size 1]

error after  32 is 
 368.2455
[torch.FloatTensor of size 1]

error after  33 is 
 391.4102
[torch.FloatTensor of size 1]

error after  34 is 
 394.3190
[torch.FloatTensor of size 1]

error after  35 is 
 401.0990
[torch.FloatTensor of size 1]

error after  36 is 
 422.3723
[torch.FloatTensor of size 1]

Solution

Probably your learning rate is too high. Try decreasing your learning rate. A too large learning rate is the most common reason for loss increasing from the first epoch.

Also your loss is very high. It is unusual to have such a high lost. You probably have a sum in your loss function, it might be wiser to replace that sum with a mean. While this makes no difference if you use the Adam optimizer, if you use simple SGD with or without momentum using a sum instead of a mean, means that you will need to tune your learning rate differently if the dimensions (or the length of your sequence processed by your lstm) of your system changes.