I want to perform regression on a dataset where the input has multiple features and the output has multiple continuous targets.
I've been looking through the sklearn documentation, but the only multi-target examples I've found have either 1) a discrete set of target labels or 2) use a heuristic algorithm like KNN instead of an optimization-based algorithm like regression. Adding regularization would also be great, but I can't find a method even for simple least-squares. This is a really simple, smooth optimization problem so I'd be shocked if it wasn't already implemented somewhere. I'd appreciate it if someone could point me in the right direction!
You can find what you are looking for here.
https://machinelearningmastery.com/multi-output-regression-models-with-python/
But it would be better to try using Keras if you have enough data (output layer without any activation.
from keras.layers import Dense, Input
from keras.models import Model
from keras.regularizers import l2
num_inputs = 10
num_outputs = 4
inp = Input((num_inputs,))
out = Dense(num_outputs, kernel_regularizer=l2(0.01))(inp)
model = Model(inp, out)
model.compile(optimizer='sgd', loss='mse', metrics=['acc','mse'])
model.summary()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_9 (InputLayer) (None, 10) 0
_________________________________________________________________
dense_7 (Dense) (None, 4) 44
=================================================================
Total params: 44
Trainable params: 44
Non-trainable params: 0
_________________________________________________________________