I can not access layers after loading the model.
I created the model as follows:
def create_model(vocab_dim, hidden_dim):
input_seq_axis1 = Axis('inputAxis1')
input_sequence_before = sequence.input_variable(shape=vocab_dim, sequence_axis=input_seq_axis1, is_sparse = use_sparse)
input_sequence_after = sequence.input_variable(shape=vocab_dim, sequence_axis=input_seq_axis1, is_sparse = use_sparse)
e=Sequential([
C.layers.Embedding(hidden_dim),
Stabilizer()
],name='Embedding')
a = Sequential([
e,
C.layers.Recurrence(C.layers.LSTM(hidden_dim//2),name='ForwardRecurrence'),
],name='ForwardLayer')
b = Sequential([
e,
C.layers.Recurrence(C.layers.LSTM(hidden_dim//2),go_backwards=True),
],name='BackwardLayer')
latent_vector = C.splice(a(input_sequence_before), b(input_sequence_after))
bias = C.layers.Parameter(shape = (vocab_dim, 1), init = 0, name='Bias')
weights = C.layers.Parameter(shape = (vocab_dim, hidden_dim), init = C.initializer.glorot_uniform(), name='Weights')
z = C.times_transpose(weights, latent_vector,name='Transpose') + bias
z = C.reshape(z, shape = (vocab_dim))
return z
Then I load the model:
def load_my_model(vocab_dim, hidden_dim):
z=load_model("models/lm_epoch0.dnn")
input_sequence_before = z.arguments[0]
input_sequence_after = z.arguments[1]
a=z.ForwardLayer
b=z.BackwardLayer
latent_vector = C.splice(a(input_sequence_before), b(input_sequence_after))
I get an error: TypeError("argument ForwardRecurrence's type SequenceOver[inputAxis1][Tensor[100]] is incompatible with the type SequenceOver[inputAxis1][SparseTensor[50000]] of the passed Variable",)
It looks like the layer referenced by name (z.ForwardLayer) represents the function from the layer immediate input. How can I calculate "latent_vector" (I need this variable to create cross entropy and loss functions to continue training) ?
Based on the error, the dimensions of your input seq is too big (5000) compared to what your ForwardLayer expects (100).
When you select the node ForwardLayer via z.ForwardLayer
, you are only select that very specific node/layer, but not the layers/nodes/rest of the computation graphs that is connected to it.
You should do a = C.combine([z.ForwardLayer.owner])
and you should be fine.