I'd like to enforce symmetry in the weights within a Variable. I really want an approximate circular symmetry. However, I could imagine either row or column enforced symmetry.
The goal is to reduce training time by reducing the number of free variables. I know my problem would like a symmetric array but I might want to include both symmetric and "free" variables. I am using conv2d now, so I believe I need to keep using it.
Here is a function that creates a kernel symmetric with respect to reflection over its center row:
def SymmetricKernels(height,width,in_channels,out_channels,name=None):
half_kernels = tf.Variable(initial_value=tf.random_normal([(height+1)//2,width,in_channels,out_channels]))
half_kernels_reversed = tf.reverse(half_kernels[:(height//2),:,:,:],[0])
kernels = tf.concat([half_kernels,half_kernels_reversed],axis=0,name=name)
return kernels
Usage example:
w = SymmetricKernels(5,5,1,1)
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())
w_ = sess.run(w)
w_[:,:,0,0]
# output:
# [[-1.299 -1.835 -1.188 0.093 -1.736]
# [-1.426 -2.087 0.434 0.223 -0.65 ]
# [-0.217 -0.802 -0.892 -0.229 1.383]
# [-1.426 -2.087 0.434 0.223 -0.65 ]
# [-1.299 -1.835 -1.188 0.093 -1.736]]
The idea is to use tf.Variable() to create only the upper half variables of the kernels (half_kernels), and then form the symmetric kernels as a concatenation of the upper half and its reflected version.
This idea can be extended to create also kernels with both left-right and up-down symmetries.