I am trying to create ASR and I am still learning so, I am just trying with a simple GRU:
MySpeechRecognition(
(gru): GRU(128, 128, num_layers=5, batch_first=True, dropout=0.5)
(dropout): Dropout(p=0.3, inplace=False)
(fc1): Linear(in_features=128, out_features=512, bias=True)
(fc2): Linear(in_features=512, out_features=28, bias=True)
)
Classifies each output as one of the possible alphabets + space + blank.
Then I use CTC Loss Function and Adam optimizer:
lr = 5e-4
criterion = nn.CTCLoss(blank=28, zero_infinity=False)
optimizer = torch.optim.Adam(net.parameters(), lr=lr)
In my training loop (I am only showing the problematic area):
output, h = mynet(specs, h)
print(output.size())
output = F.log_softmax(output, dim=2)
output = output.transpose(0,1)
# calculate the loss and perform backprop
loss = criterion(output, labels, input_lengths, label_lengths)
loss.backward()
I get this error:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-133-5e47e7b03a46> in <module>
42 output = output.transpose(0,1)
43 # calculate the loss and perform backprop
---> 44 loss = criterion(output, labels, input_lengths, label_lengths)
45 loss.backward()
46 # `clip_grad_norm` helps prevent the exploding gradient problem in RNNs / LSTMs.
/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)
/opt/conda/lib/python3.7/site-packages/torch/nn/modules/loss.py in forward(self, log_probs, targets, input_lengths, target_lengths)
1309 def forward(self, log_probs, targets, input_lengths, target_lengths):
1310 return F.ctc_loss(log_probs, targets, input_lengths, target_lengths, self.blank, self.reduction,
-> 1311 self.zero_infinity)
1312
1313 # TODO: L1HingeEmbeddingCriterion
/opt/conda/lib/python3.7/site-packages/torch/nn/functional.py in ctc_loss(log_probs, targets, input_lengths, target_lengths, blank, reduction, zero_infinity)
2050 """
2051 return torch.ctc_loss(log_probs, targets, input_lengths, target_lengths, blank, _Reduction.get_enum(reduction),
-> 2052 zero_infinity)
2053
2054
RuntimeError: blank must be in label range
I am not sure why I am getting this error. I tried changing to
labels.float()
Thanks.
Your model predicts 28 classes, therefore the output of the model has size [batch_size, seq_len, 28] (or [seq_len, batch_size, 28] for the log probabilities that are given to the CTC loss). In the nn.CTCLoss you set blank=28, which means that the blank label is the class with index 28. To get the log probabilities for the blank label you would index it as output[:, :, 28], but that doesn't work, because that index is out of range, as the valid indices are 0 to 27.
The last class in your output is at index 27, hence it should be blank=27:
criterion = nn.CTCLoss(blank=27, zero_infinity=False)