Read through a lot of articles and now I am in my braindrain end, and need a fresh perspective on the concept of mini batch. I am new to machine learning and would appreciate any advice on whether my process is correct. Here is my premise:
I have a dataset of 355 features and 8 class output. A total of 12200 data. This is a rough visualization of my neural network:
I decided on 181 neurons for Hidden layer 1, 96 neurons for Hidden Layer 2. I used ReLu activation for the hidden layers and Logistic for the output layer.
To do the mini-batch, I set my batch size to 8. So I have a total of 1525 batch with 8 dataset per batch. Here is my step:
That is the process of my mini batch. Is this correct? I mean, for the weights, do I use the weights calculated after each batch as the input weights for the next batch, or do I collect all the weights first (the starting weights will be used for all the batch), and then average the weights for all the batches? Then use the average weights as input to the next epoch?
Actually, u have to define your epoch, and each epoch should spread all your input data once at least(not only 2-7times).And after one epoch has one weight updated and repeat the steps until finish all epoch.