I am working on a project with lasagne and theano and need to create a custom layer. The output of this layer does not however depend on the size of the input, but on the values of the input... I know that keras (only with the tensorflow backend) offers the possibility of lambda layers, and I managed to write an expression which allowed me to have the output depending on the values of the input. But I don't know how or even if it is possible to do so using lasagne and theano.
For example: if my input tensor has a fixed size of 100 values, but I know that at the end there could be some 0 values, which do not influence the output of the network at all, how can I remove those values and let only the values with information go further to the next layer? I would like to minimize the space requirements of the network :)
Is there a possibility to have a layer in lasagne like that? If so, how should I write the get_output_shape_for() method? If not, I'll switch to keras and tensorflow :D
Thanks in advance!
Thanks to Jan Schlüter for providing me with the answer here:
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/lasagne-users/ucjNayfhSu0
To summarize:
1) Yes, it is possible to have a lasagne layer whose output shape depends on the input values (instead of the input shape) and
2) You must write "None" in the dimensions which do hot have a fixed compile-time shape (so the changed dimensions that depend on the input values).
Regarding the example:
You can compute the output shape first, then create a new tensor with the shape of the length of non-zero entries in the original tensor and then fill the new tensor with the non-zero values (e.g. using the theano.tensor.set_subtensor function). However, I don't know if this is the optimal solution to achieve this result...