pythonmultilabel-classificationdbnnolearn

nolearn for multi-label classification


I tried to use DBN function imported from nolearn package, and here is my code:

from nolearn.dbn import DBN
import numpy as np
from sklearn import cross_validation

fileName = 'data.csv'
fileName_1 = 'label.csv'

data = np.genfromtxt(fileName, dtype=float, delimiter = ',')
label = np.genfromtxt(fileName_1, dtype=int, delimiter = ',')

clf = DBN(
    [data, 300, 10],
    learn_rates=0.3,
    learn_rate_decays=0.9,
    epochs=10,
    verbose=1,
    )

clf.fit(data,label)
score = cross_validation.cross_val_score(clf, data, label,scoring='f1', cv=10)
print score

Since my data has the shape(1231, 229) and label with the shape(1231,13), the label sets looks like ([0 0 1 0 1 0 1 0 0 0 1 1 0] ...,[....]), when I ran my code, I got the this error message: bad input shape (1231,13). I wonder two problem might happened here:

  1. DBN does not support multi-label classification
  2. my label is not suitable to be used in DBN fit function.

Solution

  • As mentioned by Francisco Vargas, nolearn.dbn is deprecated and you should use nolearn.lasagne instead (if you can).

    If you want to do multi-label classification in lasagne, then you should set your regression parameter to True, define a validation score and a custom loss.

    Here's an example:

    import numpy as np
    import theano.tensor as T
    from lasagne import layers
    from lasagne.updates import nesterov_momentum
    from nolearn.lasagne import NeuralNet
    from nolearn.lasagne import BatchIterator
    from lasagne import nonlinearities
    
    # custom loss: multi label cross entropy
    def multilabel_objective(predictions, targets):
        epsilon = np.float32(1.0e-6)
        one = np.float32(1.0)
        pred = T.clip(predictions, epsilon, one - epsilon)
        return -T.sum(targets * T.log(pred) + (one - targets) * T.log(one - pred), axis=1)
    
    
    net = NeuralNet(
        # customize "layers" to represent the architecture you want
        # here I took a dummy architecture
        layers=[(layers.InputLayer, {"name": 'input', 'shape': (None, 1, 229, 1)}),
    
                (layers.DenseLayer, {"name": 'hidden1', 'num_units': 20}),
                (layers.DenseLayer, {"name": 'output', 'nonlinearity': nonlinearities.sigmoid, 'num_units': 13})], #because you have 13 outputs
    
        # optimization method:
        update=nesterov_momentum,
        update_learning_rate=5*10**(-3),
        update_momentum=0.9,
    
        max_epochs=500,  # we want to train this many epochs
        verbose=1,
    
        #Here are the important parameters for multi labels
        regression=True,  
    
        objective_loss_function=multilabel_objective,
        custom_score=("validation score", lambda x, y: np.mean(np.abs(x - y)))
    
        )
    
    net.fit(X_train, labels_train)