I think something wrong with my Keras multiple outputs coding, which causes a high loss comparing with the Sequential model. Please help me out which part of it is wrong.
import os, random, string, pandas, math, numpy
import tensorflow as tf
from tensorflow import keras
Training data:
feature_data = [] # common feature data
label_data = [] # for multiple outputs
single_data = [] # for single output
size = 10000
features = ['x1', 'x2']
labels = ['y1', 'y2']
for i in range(size):
a = random.random()
b = random.random()
c = math.sin(a)
d = math.cos(b)
feature = [a, b]
label = [c, d]
feature_data.append(feature)
label_data.append(label)
single_data.append(c)
This is my single output model, which is working well: loss < 2e-05
single = keras.Sequential([
keras.layers.Dense(2, input_shape=(2,), activation=tf.nn.softmax),
keras.layers.Dense(4, activation=tf.nn.softmax),
keras.layers.Dense(1)])
optimizer = tf.optimizers.RMSprop(learning_rate=0.001)
single.compile(loss='mse', optimizer=optimizer, metrics=['mae'])
single.fit(x=feature_data, y=single_data, epochs=100, batch_size=100)
This should be the identical multiple output model, but the losses are really high: 0.1
def build_model():
input_shape=(2, )
inputs = keras.Input(shape=input_shape)
outputs = []
for label in labels:
u = keras.layers.Dense(2, input_shape=input_shape, activation=tf.nn.softmax)(inputs)
v = keras.layers.Dense(4, activation=tf.nn.softmax)(u)
w = keras.layers.Dense(1, name=label)(v)
outputs.append(w)
model = keras.Model(inputs = inputs, outputs = outputs)
optimizer = tf.optimizers.RMSprop(learning_rate=0.001)
model.compile(loss='mse', optimizer=optimizer, metrics=['mae'])
return model
model = build_model()
model.fit(x=feature_data, y=label_data, epochs=100, batch_size=100)
I guess that something wrong with the input layer, or the label data format, but still have no idea how to fix it. Please help.
I was not paying attention.
Softmax is for picking one out of several possible outputs. You are doing continuous level outputs, not one-hot encoding.
I think your second model should work if you pick the right activation function at the right Dense layers.
Looking at your data, you are predicting sin of (0->1) and cos of (0->1). From the chart, these are both very flat lines within the range of sin & cos. This is such a small model that maybe tanh or sigmoid on any or all Dense layer should do the job.