I am trying to predict vehicle trajectory, using t-30 data sequence to predict until t+30 trajectory. The data I have created is an neighbour occupancy matrix for each frames for each cars, this data is 13x3x1x30x138634 where 13x3 the grid, 1 is the channel, 30 is sequence length and 138634 is the observation number. For the target vehicle I have 2 arrays of 30x3x138634 where 30 is the sequence length, 3 is (x,y,v) for the target vehicle and 138634 is the observation number. So neighbour data is SSCTB and target data is TCB. The error I am getting is:
Error forming mini-batch for network input "sequenceinput". Data interpreted with format "SSCTB". To specify a different format, use the InputDataFormats option.
Error in
main (line 488)
trainnedNet = trainnet(TrainDS,net,"l2loss",options);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Caused by:
Batch dimension of datastore must match the format batch dimension (5).
I have attached the code but can't attach *.mat files they are too large even when zipped. You can simply assume in the code
input_neigh_odd = zeros(13x3x1x30x138634);
input_target_odd_centered = zeros(30x3x138634);
input_target_even_centered = zeros(30x3x138634);
The code part is,
load("neighInputOdd.mat")
load("targetInputOddCentered.mat")
load("targetInputEvenCentered.mat")
numSamples = 138634;
shuffledIdx = randperm(numSamples);
nTrain = floor(0.70 * numSamples);
nVal = floor(0.15 * numSamples);
nTest = numSamples - nTrain - nVal;
trainIdx = shuffledIdx(1:nTrain);
valIdx = shuffledIdx(nTrain+1:nTrain+nVal);
testIdx = shuffledIdx(nTrain+nVal+1:end);
X1Train = {permute(input_neigh_odd(:,:,:,:,trainIdx), [1 2 3 5 4])};
X1Val = {permute(input_neigh_odd(:,:,:,:,valIdx), [1 2 3 5 4])};
X1Test = {permute(input_neigh_odd(:,:,:,:,testIdx), [1 2 3 5 4])};
X2Train = {input_target_odd_centered(:,:,trainIdx)};
X2Val = {input_target_odd_centered(:,:,valIdx)};
X2Test = {input_target_odd_centered(:,:,testIdx)};
YTrain = {input_target_even_centered(:,:,trainIdx)};
YVal = {input_target_even_centered(:,:,valIdx)};
YTest = {input_target_even_centered(:,:,testIdx)};
save("TrainValTest\X1Train.mat","X1Train","-v7.3")
save("TrainValTest\X2Train.mat","X2Train","-v7.3")
save("TrainValTest\YTrain.mat","YTrain","-v7.3")
save("TrainValTest\X1Val.mat","X1Val","-v7.3")
save("TrainValTest\X2Val.mat","X2Val","-v7.3")
save("TrainValTest\YVal.mat","YVal","-v7.3")
save("TrainValTest\X1Test.mat","X1Test","-v7.3")
save("TrainValTest\X2Test.mat","X2Test","-v7.3")
save("TrainValTest\YTest.mat","YTest","-v7.3")
matFiles = dir(fullfile('TrainValTest', '*.mat'));
for i = 1:numel(matFiles)
filePath = fullfile(matFiles(i).folder, matFiles(i).name);
[~, name, ~] = fileparts(matFiles(i).name);
varName = [name 'DS'];
ds = signalDatastore(filePath);
assignin('base', varName, ds);
end
TrainDS = combine(X1TrainDS,X2TrainDS,YTrainDS);
ValDS = combine(X1ValDS,X2ValDS);
alpha = 0.001;
options = trainingOptions( ...
"adam", ...
"InputDataFormats", {'SSCTB', 'TCB'},...
Plots="training-progress", ...
ValidationData = {ValDS,YValDS}, ...
ValidationFrequency = 100, ...
ValidationPatience = 5, ...
ExecutionEnvironment = "parallel-auto",...
MaxEpochs = 15,...
InitialLearnRate=alpha,...
MiniBatchSize=256,...
L2Regularization=0.0001,...
Verbose=false);
net=dlnetwork;
numHiddenUnits=64;
numNeurons=128;
layers1=[
sequenceInputLayer([13 3 1])
convolution2dLayer([3 3],16,'Padding','same')
batchNormalizationLayer
reluLayer
convolution2dLayer([3 3],32,'Padding','same')
batchNormalizationLayer
reluLayer
bilstmLayer(64,'OutputMode','sequence','name','bilstm1')
concatenationLayer(1,2,'Name','cat')
fullyConnectedLayer(3)
];
layers2=[
sequenceInputLayer(3)
bilstmLayer(32,'OutputMode','sequence','name','bilstm2')
];
net=addLayers(net,layers1);
net=addLayers(net,layers2);
net=connectLayers(net,'bilstm2','cat/in2');
trainnedNet = trainnet(TrainDS,net,"l2loss",options);
The error happens because you're wrapping your entire dataset in a single cell, so MATLAB thinks you have only one observation. For training, MATLAB needs to treat each sample as a separate item along the batch dimension , 5th for SSCTB and 3rd for TCB. To fix it, use arrayDatastore with the correct iteration dimensions:
X1 = permute(input_neigh_odd, [1 2 3 5 4]); % SSCTB
X2 = input_target_odd_centered; % TCB
Y = input_target_even_centered; % TCB
X1TrainDS = arrayDatastore(X1(:,:,:,:,trainIdx), 'IterationDimension', 5);
X2TrainDS = arrayDatastore(X2(:,:,trainIdx), 'IterationDimension', 3);
YTrainDS = arrayDatastore(Y(:,:,trainIdx), 'IterationDimension', 3);
X1ValDS = arrayDatastore(X1(:,:,:,:,valIdx), 'IterationDimension', 5);
X2ValDS = arrayDatastore(X2(:,:,valIdx), 'IterationDimension', 3);
YValDS = arrayDatastore(Y(:,:,valIdx), 'IterationDimension', 3);
TrainDS = combine(X1TrainDS, X2TrainDS, YTrainDS);
ValDS = combine(X1ValDS, X2ValDS, YValDS);
Now trainnet will correctly interpret the batch dimensions, and the error should go away.