[SOLVED] Tuning size parameter for neural network

Tuning size parameter for neural network

I want to fit a neural network model using caret package.There are 208 predictors all of which are important and cannot be discarded. The maximum value that I can give to the size parameter is 4 beyond which I get an error saying that there are too many weights.

> ctrl<-trainControl(method = 'cv',number = 5)
> my.grid <- expand.grid(.decay = 0.1, .size =5)
> nn.fit <- train(train_predictors,train_responses[["r2c1"]],method = "nnet",algorithm = 'backprop', tuneGrid = my.grid,trace=F, linout = TRUE,trControl = ctrl)
Something is wrong; all the RMSE metric values are missing:
      RMSE        Rsquared        MAE     
 Min.   : NA   Min.   : NA   Min.   : NA  
 1st Qu.: NA   1st Qu.: NA   1st Qu.: NA  
 Median : NA   Median : NA   Median : NA  
 Mean   :NaN   Mean   :NaN   Mean   :NaN  
 3rd Qu.: NA   3rd Qu.: NA   3rd Qu.: NA  
 Max.   : NA   Max.   : NA   Max.   : NA  
 NA's   :1     NA's   :1     NA's   :1    
Error: Stopping
In addition: Warning messages:
1: model fit failed for Fold1: decay=0.1, size=5 Error in nnet.default(x, y, w, ...) : too many (1051) weights

2: model fit failed for Fold2: decay=0.1, size=5 Error in nnet.default(x, y, w, ...) : too many (1051) weights

3: model fit failed for Fold3: decay=0.1, size=5 Error in nnet.default(x, y, w, ...) : too many (1051) weights

4: model fit failed for Fold4: decay=0.1, size=5 Error in nnet.default(x, y, w, ...) : too many (1051) weights

5: model fit failed for Fold5: decay=0.1, size=5 Error in nnet.default(x, y, w, ...) : too many (1051) weights

6: In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,  :
  There were missing values in resampled performance measures.

The model performs very badly with 4 neurons(size=4).What can I do to make the model work if I want to have more than 5 neurons?

Solution

There are other parameters in the tuning grid that you can specify for the nnet method. The available parameters for each method are available online but difficult to find. Here's my example using mxnet for adam nn:

mxnet_grid_A2 = expand.grid(layer1 = c(10, 12),   
                           layer2 = c(4, 6),
                           layer3 = 2,
                           learningrate = c(0.001, 0.0001),
                           dropout = c(0, 0.2)
                           beta1 = .9,
                           beta2 = 0.999,
                           activation = 'relu')