So, i am new to NLP, and i am trying to train a text classifier using the spacy_transformers. this code has been shown to run, but it throws up errors on my computer. As a side note, could it be caused by the fat that i am running it on a cpu?
def train_classifier(n_epoch:int=5,
train_data:list=None,
val_text:tuple=None,
val_label:list=None,
batch_size:int=32,
lr:float=1e-3):
train_stats = []
dropout = decaying(0.2, 0.1, 0.3) # Gradually decrease dropout rate from 0.2 to 0.1
# Cyclic triangular rate (https://arxiv.org/abs/1506.01186)
learn_rates = cyclic_triangular_rate(
lr / 3, lr * 3, 2 * len(train_data) // batch_size
)
for epoch in range(n_epoch):
random.shuffle(train_data)
batches = minibatch(train_data, size=batch_size)
losses = {}
for batch in batches:
optimizer.trf_lr = next(learn_rates)
texts, cats = zip(*batch)
nlp.update(
texts,
cats,
drop = next(dropout),
sgd = optimizer,
losses=losses)
and then when i pass the function
train_classifier(n_epoch=10, train_data=train_data, val_text=val_text, val_label=val_label, batch_size=32, lr=2e-6)
i get the following error
<ipython-input-55-5bb071ef310c> in train_classifier(n_epoch, train_data, val_text, val_label, batch_size, lr)
6 lr:float=1e-3):
7 train_stats = []
----> 8 dropout = decaying(0.2, 0.1, 0.3) # Gradually decrease dropout rate from 0.2 to 0.1
9 # Cyclic triangular rate (https://arxiv.org/abs/1506.01186)
10 learn_rates = cyclic_triangular_rate(
NameError: name 'decaying' is not defined
I'm not at all familiar with spacy, but after a few google searches it looks like this function may be dependent on spacy's util.decaying function. Without the decaying function loaded into memory, this train_classifier function will throw a NameError.
The code for the decaying function is as follows and can be found here.
def decaying(start, stop, decay):
"""Yield an infinite series of linearly decaying values."""
curr = float(start)
while True:
yield max(curr, stop)
curr -= decay