It's not about loading the weights of a model. I am trying to see the weights of layers in loaded efficientnet-b3 model in the torch.
os.system('pip install efficientnet_pytorch')
from efficientnet_pytorch import EfficientNet
MODEL_NAME = 'efficientnet-b3'
effnet = EfficientNet.from_pretrained(MODEL_NAME)
effnet.modules # this works, but only gives the module names
effnet.weights # doesn't work
effnet.layers # doesn't work
effnet.modules[1] # doesn't work, second module is batch norm ._bc0
I would like some functionality to replicate the following TF code in torch, access weights of the first batch norm layer
from tensorflow.keras.applications import EfficientNetB3
base_model = EfficientNetB3(weights="imagenet")
base_model.trainable_variables[1].numpy() # indexing with 1 gives weights of conv layer
Output for the above code:
<tf.Variable 'stem_bn/gamma:0' shape=(40,) dtype=float32, numpy=
array([ 0.1913209 , 2.7074034 , 9.623442 , 2.5562265 , 3.127593 ,
4.348222 , 2.4381876 , 3.4623973 , 3.6115906 , 4.1241236 ,
2.18851 , 8.9716835 , 0.7232651 , 0.6261555 , 9.050293 ,
7.9233327 , 0.47725916, 3.4991856 , 5.334402 , 4.843143 ,
1.4122163 , 1.953061 , 8.150878 , 5.0044165 , 2.3806598 ,
4.2976685 , 2.2239766 , 0.551327 , 7.799995 , 3.3823645 ,
1.8910869 , 4.0793633 , 0.73215246, 3.4526935 , 10.874565 ,
2.0920732 , 6.272054 , 3.6823177 , 4.2152214 , 3.4319222 ],
dtype=float32)>
You can access the weights of a model by calling model.named_parameters().
In your case, calling effnet.named_parameters() returns a list of 2-tuples, where the first item in the tuple is the name of the parameter, and the second item is the actual parameter tensor.
Here is a sample extract so you can see what I mean:
[('_conv_head.weight', Parameter containing:
tensor([[[[ 0.0074]],
[[-0.0904]],
[[-0.0091]],
...,
[[-0.0504]],
[[ 0.1140]],
[[-0.0710]]],
[[[ 0.0635]],
[[ 0.0290]],
[[-0.0583]],
...,
[[-0.0813]],
[[-0.0230]],
[[-0.1120]]],
[[[ 0.0873]],
[[-0.0174]],
[[-0.0170]],
...,
[[-0.0014]],
[[ 0.0323]],
[[ 0.0569]]],
...,
[[[-0.0218]],
[[ 0.0292]],
[[ 0.0267]],
...,
[[-0.0044]],
[[-0.0463]],
[[-0.0257]]],
[[[-0.0209]],
[[ 0.0279]],
[[ 0.0094]],
...,
[[-0.1759]],
[[-0.0702]],
[[-0.0902]]],
[[[-0.0488]],
[[ 0.0276]],
[[-0.0174]],
...,
[[-0.0391]],
[[-0.0268]],
[[-0.0205]]]], requires_grad=True)),
('_bn1.weight', Parameter containing:
tensor([2.3115, 2.0343, 2.0015, ..., 1.7868, 2.3552, 1.8885],
requires_grad=True)),
('_bn1.bias', Parameter containing:
tensor([-1.8066, -1.4178, -1.3111, ..., -1.0494, -1.8520, -1.1798],
requires_grad=True)),
('_fc.weight', Parameter containing:
tensor([[-0.0157, -0.0483, 0.0139, ..., -0.0201, -0.0092, -0.0752],
[-0.0100, -0.0575, 0.0328, ..., -0.0362, -0.0534, -0.0041],
[-0.0083, 0.0142, -0.0006, ..., 0.0151, -0.0033, 0.0500],
...,
[-0.0840, -0.0217, -0.0354, ..., -0.0620, 0.0143, 0.0786],
[-0.0956, -0.0169, 0.0738, ..., 0.1063, -0.0742, 0.0036],
[ 0.0485, 0.0470, 0.1002, ..., -0.0832, 0.1081, 0.0145]],
requires_grad=True)),
('_fc.bias', Parameter containing:
tensor([-1.8788e-04, -2.1204e-02, -2.2974e-02, -3.5067e-02, -4.2469e-02,
-4.0472e-02, -3.3574e-02, 7.5904e-03, -1.3298e-02, -1.3364e-02,
-4.7947e-02, -6.8513e-02, -5.1592e-02, -4.0660e-02, -2.1086e-02,
-5.7097e-02, -8.5144e-02, -4.0252e-02, 1.5397e-02, -2.6116e-02,
-7.2120e-02, -4.8167e-02, -4.5482e-02, -5.7588e-02, -6.5176e-02,
-4.3350e-02, -6.1431e-03, -3.3575e-02, -1.6232e-02, 4.4864e-03,
-7.7549e-02, -6.1085e-02, -3.7735e-02, -4.0341e-02, 1.7911e-03,
-7.5653e-02, 3.0368e-02, -3.7621e-02, -1.5108e-02, -1.8987e-02,
-7.0831e-02, -4.8989e-02, -4.6129e-02, -3.8295e-02, -2.3450e-02,
-7.5764e-02, -9.7884e-03, -2.0963e-02, -5.0398e-02, -3.1158e-02,
-4.3633e-02, 2.1600e-02, 2.2160e-02, 9.6163e-03, -3.5956e-02,
2.3973e-03, -3.3229e-02, -5.7656e-02, -1.0077e-02, -1.7506e-02,
3.1533e-02, 4.6831e-02, -1.2585e-02, -5.1598e-03, 2.8223e-02,
-8.0142e-03, -3.1187e-03, -5.5687e-02, -1.6465e-02, -6.9663e-02,
-1.0505e-02, -1.2698e-02, -4.9928e-02, 2.3078e-02, -3.1174e-02,
-9.6878e-03, 3.8667e-02, 1.0131e-02, 2.0751e-02, 1.1730e-03,
-2.4952e-02, -6.8574e-02, -2.2635e-02, -8.7558e-02, -7.1596e-02,
-9.8193e-04, -3.9943e-02, 1.3720e-02, -1.0468e-02, -8.8728e-03,
-6.1661e-02, -6.7819e-02, -4.3548e-02, -6.7823e-02, -1.1364e-01,
-8.0226e-02, -3.1601e-02, -4.1108e-02, -1.0578e-01, -2.4910e-02,
-3.6718e-02, -6.0854e-03, -1.7552e-02, 4.8444e-02, -1.0772e-01,
-8.1519e-02, 1.1975e-02, -5.9317e-02, -4.8866e-02, -1.0751e-01,
-4.0855e-02, -2.0209e-02, 2.0903e-02, -4.4489e-02, 4.7728e-03,
-1.0581e-01, -6.0772e-02, -5.2845e-02, 5.1716e-02, -6.6787e-02,
-6.2687e-02, 8.3072e-03, -7.7149e-03, -8.9555e-02, -4.8636e-02,
-7.7438e-02, -2.0582e-02, -3.4283e-02, -6.6013e-02, -8.3553e-02,
-1.7237e-02, 1.0309e-02, -1.3888e-02, -8.5651e-02, -1.0593e-02,
-8.5910e-02, -4.4669e-02, -4.7769e-02, -5.6502e-02, -3.7197e-02,
-5.9685e-02, -6.1591e-02, -3.5832e-02, -3.9733e-02, -2.6279e-03,
-7.5230e-03, -4.8190e-02, -5.2629e-02, 1.3794e-03, -6.2900e-02,
1.2298e-02, 3.3339e-02, -3.2597e-02, 2.7527e-02, -1.2776e-02,
4.9703e-02, 1.7361e-02, 2.0815e-02, -3.7160e-02, 3.7085e-02,
-3.6573e-02, 6.2998e-02, 6.3216e-02, 3.9644e-02, 2.1697e-02,
-7.6096e-02, -3.1489e-02, -4.0166e-02, 3.0928e-02, 1.8492e-02,
3.6115e-03, 6.6122e-02, 3.6215e-02, 1.1731e-02, 6.3609e-02,
-5.1814e-02, -2.1708e-02, 2.3418e-02, 1.1752e-01, -5.7509e-05,
-9.9666e-03, 3.7792e-02, 5.2852e-02, 2.7368e-02, 8.6873e-02,
5.4690e-03, 1.4777e-02, 1.8022e-02, 1.4611e-02, 8.3892e-02,
-2.4939e-02, 9.1274e-02, 6.7359e-04, 1.2745e-02, -3.5555e-02,
1.3300e-01, 4.2316e-02, 3.8856e-02, -1.3746e-02, 7.2174e-02,
-3.3013e-02, 2.3095e-02, 4.3574e-02, 9.0962e-02, -2.4382e-03,
1.9608e-02, -9.1173e-03, 5.3899e-02, 4.8285e-02, 8.5972e-02,
-1.0665e-02, 3.8411e-02, 3.8558e-03, 5.2799e-03, 1.4861e-02,
3.9330e-02, 7.3648e-03, 8.2185e-02, 5.4206e-02, 5.7331e-03,
6.0913e-02, -2.7097e-02, 9.5242e-03, 5.7349e-02, -8.2799e-03,
3.3900e-02, -4.3817e-02, 7.2375e-03, -7.7864e-03, 6.6191e-03,
-1.6398e-03, 3.7898e-03, 2.0504e-02, -2.7386e-02, 3.3018e-02,
2.3809e-02, 6.7705e-02, 7.4084e-02, 4.5319e-02, 4.4979e-02,
2.6763e-02, 2.5106e-02, 5.7135e-02, 4.2436e-02, -5.5549e-02,
8.4225e-02, -1.8866e-02, 5.4292e-02, 7.3890e-02, 4.6438e-02,
2.8466e-02, 9.7005e-02, -1.9982e-02, 5.4305e-02, 5.3359e-02,
4.3933e-03, 1.2966e-02, -1.3056e-02, 2.8389e-02, 6.9528e-03,
-9.5808e-03, 1.3317e-02, -5.5829e-02, 4.7666e-03, 3.1541e-02,
2.3011e-02, 4.1077e-02, 5.9078e-02, -5.6244e-02, -1.7187e-02,
4.5811e-02, 3.0973e-02, 1.8721e-02, 8.7556e-03, -1.3825e-02,
1.6460e-03, -6.1395e-02, -1.6215e-02, -9.6606e-03, -3.0871e-02,
2.9623e-02, 4.9153e-02, 4.9580e-02, -1.5432e-02, 4.9673e-02,
3.5004e-02, -3.5870e-02, -4.6010e-02, -3.7892e-02, -5.9807e-03,
7.9569e-03, -8.6313e-02, -2.0522e-02, -2.7805e-02, -4.1220e-02,
-4.9287e-02, -9.1146e-03, -2.2379e-02, -3.7943e-02, -2.0446e-02,
-6.7551e-02, -2.6903e-02, 3.5022e-03, -4.7364e-02, -1.0330e-01,
-6.1197e-02, -1.9515e-02, -6.4015e-02, -3.7570e-02, -4.0806e-02,
-1.7590e-02, -6.0645e-02, -3.7628e-02, 2.2510e-02, 6.6436e-02,
-3.2913e-02, -7.2844e-02, -8.4280e-02, -9.0194e-04, -3.3313e-02,
-5.0213e-02, -6.0932e-02, -6.3614e-02, -6.2003e-02, -1.8194e-02,
-7.3311e-02, -6.6968e-02, -1.9100e-02, -6.7069e-02, -9.9181e-02,
-2.4951e-02, -2.4315e-02, -2.8859e-02, 2.5487e-02, 1.7767e-03,
-4.7624e-02, -1.6528e-02, -2.6242e-02, -2.0435e-02, -4.6075e-03,
1.4161e-02, -1.7872e-02, -2.7038e-02, -2.0696e-02, -3.1628e-02,
3.1667e-02, 1.2643e-02, 3.3634e-02, -2.2169e-02, -1.3101e-02,
-2.6296e-02, -2.9903e-02, -5.1367e-02, -2.5846e-02, 1.1791e-02,
-2.0295e-02, 2.2274e-02, 3.4522e-02, 9.1384e-02, 5.7635e-02,
-3.7340e-02, 4.5483e-02, 9.2709e-03, -3.4176e-03, -1.8723e-02,
4.7653e-02, -4.3498e-02, 1.5783e-02, -2.6597e-03, 4.8437e-02,
5.4055e-03, 5.2496e-02, -7.5699e-02, -7.2362e-03, -2.4031e-02,
-3.8313e-02, -5.7455e-02, -3.2797e-02, -1.0619e-02, -3.5333e-02,
6.8303e-02, 4.2202e-02, 1.2961e-02, -1.0357e-02, -7.2061e-02,
3.5756e-02, 1.0100e-02, -6.8939e-03, -3.2524e-03, 1.9981e-02,
-7.9577e-02, -9.9632e-03, -6.4924e-02, -7.7214e-02, -4.2186e-03,
7.7079e-03, -6.8280e-02, -9.0313e-02, 5.5780e-02, 4.0597e-02,
1.5803e-02, 7.6331e-02, 8.0219e-02, -4.2492e-02, -5.0285e-02,
2.8535e-02, -5.1758e-02, -5.5653e-02, -2.6481e-02, 8.4919e-02,
3.3701e-02, -4.7973e-02, 4.3328e-02, 5.4467e-02, -3.5105e-02,
-2.8679e-02, 3.2789e-02, -3.0762e-02, -5.7570e-03, 6.3135e-02,
2.6109e-02, 4.5541e-02, 1.5574e-02, 3.4254e-02, 2.1206e-02,
8.7765e-03, -1.5536e-02, 8.7542e-02, 2.8484e-02, 7.4923e-02,
2.7793e-03, 3.0007e-02, 4.5130e-02, 3.4480e-02, 1.4038e-02,
1.0104e-01, 8.9144e-03, -6.6594e-03, 2.3476e-02, 2.1639e-02,
8.8072e-02, 3.4813e-02, -6.1209e-02, -1.9577e-02, -1.3342e-02,
8.3109e-02, -5.9933e-02, 4.5758e-02, 8.2524e-02, -3.3844e-02,
-2.8108e-02, -1.2548e-02, 2.2878e-03, -3.2685e-03, 4.4044e-02,
8.5652e-02, 3.7278e-03, 7.4127e-02, -4.7680e-02, -1.0711e-02,
-1.9743e-03, 2.8187e-02, 3.1251e-02, 1.3193e-01, 1.3656e-02,
-8.9823e-03, -8.9561e-03, -3.4255e-02, 2.8345e-02, 3.1666e-02,
5.3442e-02, 2.4717e-02, -1.8101e-02, 2.4845e-02, -1.5233e-02,
1.0711e-01, -2.0983e-02, -4.5709e-02, -8.3504e-04, -2.5989e-02,
1.4240e-02, 1.3953e-02, -1.6013e-02, 9.2497e-04, -3.4750e-02,
-3.9424e-02, 7.6613e-02, -1.2504e-02, 1.2788e-01, 7.7914e-02,
6.1930e-02, 1.4219e-03, -2.5443e-02, 6.1517e-03, 6.1918e-02,
-2.7085e-02, 7.3593e-03, -7.3484e-02, 2.8937e-02, 9.8165e-02,
2.0922e-03, 4.2529e-02, -1.1309e-02, -4.9490e-02, 1.6751e-02,
1.0809e-02, 2.3618e-02, 2.0515e-02, 3.3854e-02, -1.1171e-01,
-9.9522e-03, -2.2146e-02, 1.3169e-02, 1.1341e-01, 6.5784e-02,
1.1520e-01, -1.2588e-02, -7.9052e-02, -1.4347e-02, 4.7605e-02,
3.4523e-02, -4.4671e-02, 1.1775e-02, 1.0015e-01, 2.6564e-02,
-1.3457e-02, -4.4883e-02, 3.8352e-02, 1.2466e-02, -1.1996e-03,
1.0156e-01, -9.3025e-02, 1.9178e-02, -3.7339e-02, -3.1499e-02,
1.3383e-02, -2.9213e-02, -4.8970e-02, -4.5686e-02, 2.8906e-02,
-5.9064e-02, -1.5512e-03, 1.9657e-02, 7.8270e-03, -4.0602e-02,
1.9702e-02, 4.0297e-02, -1.9938e-02, -1.2144e-02, -2.3456e-02,
-5.3594e-02, -4.2403e-02, 7.4747e-02, 1.2299e-02, -2.0902e-02,
-5.5567e-02, 1.4648e-02, 1.4435e-03, 1.0079e-01, 9.2539e-02,
-1.0607e-02, -1.4108e-02, 9.1721e-03, 1.1072e-02, 2.9608e-02,
-7.6078e-03, 2.5653e-02, -1.1136e-02, 7.2409e-02, -5.1752e-02,
8.0248e-02, -6.1112e-02, 6.1196e-02, -2.2555e-02, 8.3575e-02,
-3.1096e-02, -4.8845e-03, 7.4904e-02, -6.5950e-03, 1.9850e-02,
-3.0121e-02, 3.6329e-02, -4.4467e-03, 8.3675e-02, -7.4437e-02,
-4.0771e-02, -2.8450e-02, 8.7870e-02, -4.0946e-02, -3.8329e-03,
-6.9315e-02, -1.0016e-02, -2.0778e-02, 8.0915e-02, 3.9063e-02,
-1.5094e-02, 3.3627e-03, -2.3838e-02, 3.1394e-02, 1.2513e-02,
1.3142e-01, 3.0603e-02, -2.2055e-03, 2.0502e-02, 1.0135e-01,
7.1754e-02, 9.7944e-03, 3.7959e-02, 1.3989e-02, 1.7989e-02,
-3.4136e-03, 3.9650e-02, -3.4606e-02, -2.8792e-03, -5.2186e-03,
-3.3676e-02, 5.8599e-02, 6.2828e-02, 4.3374e-02, 2.6202e-02,
9.4092e-02, -3.8893e-03, 9.6722e-02, -6.6398e-02, -5.4730e-02,
-4.2413e-02, 2.9454e-02, -1.1035e-02, -1.3873e-02, 3.5274e-02,
-5.2795e-03, 7.2810e-03, 5.6811e-03, 2.5880e-02, -1.6221e-02,
-4.2152e-02, 2.0760e-02, 1.4120e-02, 7.2473e-03, 6.9235e-03,
-6.1877e-02, 8.1591e-03, 5.1506e-02, 4.0437e-02, 1.4943e-02,
-2.9214e-02, 7.7097e-02, -3.1952e-02, 3.0927e-03, -6.7025e-04,
6.2347e-02, 8.4001e-03, 3.2072e-02, 5.1373e-02, -2.0174e-02,
1.1651e-02, 1.1074e-02, 1.7667e-02, -8.6863e-03, 3.0662e-02,
-1.7535e-02, -5.4594e-02, -1.0278e-02, -6.7767e-02, 1.8709e-02,
4.3460e-02, 2.7637e-02, 5.1533e-02, -3.5134e-02, 3.0937e-02,
1.0212e-02, 3.0756e-02, 6.7154e-02, 3.9204e-02, 6.0210e-02,
-6.5048e-02, 7.2162e-02, 4.5481e-02, 4.0012e-02, -2.6958e-02,
2.3580e-03, 1.9551e-02, 1.4279e-02, 6.8494e-02, 2.1821e-02,
-3.8355e-02, -5.8506e-02, -7.3684e-02, -7.3094e-03, -4.3607e-02,
-2.3467e-03, -8.3147e-03, 6.3290e-02, -1.1461e-02, -1.4600e-03,
6.5386e-02, 9.3397e-03, -4.8493e-02, -2.3290e-02, 4.6487e-02,
1.0229e-01, 4.5283e-03, 4.7067e-02, 1.0678e-02, 5.5262e-02,
7.6263e-03, -3.7516e-02, 1.8642e-02, 1.2551e-02, -7.5747e-02,
-6.9164e-03, 9.2134e-03, -7.1654e-02, -3.2144e-02, 4.4780e-02,
1.6665e-02, 5.4213e-02, -5.7801e-02, -4.3401e-05, 9.2516e-02,
2.4602e-02, -2.6873e-02, 6.9050e-02, -1.1062e-02, -6.5441e-02,
3.2812e-02, 1.6676e-02, -3.6145e-02, 6.2055e-02, -5.5214e-02,
-2.6646e-02, 1.5051e-01, -1.4609e-02, 7.3359e-02, -1.1975e-02,
2.5550e-03, 2.8461e-02, 6.1058e-02, 1.5934e-02, 1.1280e-02,
-5.7301e-02, -8.1453e-02, -1.3112e-02, 1.4247e-02, 2.6493e-02,
-3.2159e-02, -1.2888e-02, 3.9029e-02, -3.8758e-02, 1.1523e-01,
5.7670e-02, -3.0118e-02, -1.3478e-02, 5.0629e-02, -1.2562e-02,
-3.1457e-02, -7.3343e-03, -4.2227e-02, -5.6207e-02, -2.7644e-02,
-2.3450e-02, 3.6726e-02, -1.7531e-02, 5.1430e-02, 8.6157e-02,
3.9421e-02, -3.5995e-02, -1.9596e-03, 1.4217e-02, 8.6251e-02,
9.4631e-04, -8.1190e-02, 9.1071e-02, 2.4760e-02, -1.1475e-02,
2.2392e-02, 5.3320e-02, 5.3901e-04, -4.8530e-02, -4.3654e-03,
-5.6498e-02, -1.8796e-02, 7.8893e-02, 4.2227e-02, 5.3552e-02,
7.8111e-02, 2.7515e-02, -1.6096e-02, 1.2153e-02, 6.6272e-03,
-4.8895e-02, 5.0869e-02, 6.7843e-02, 6.1599e-02, -1.1996e-02,
-9.4545e-03, 1.0977e-01, 5.2409e-02, -4.7280e-02, 3.6631e-02,
-2.9781e-02, -4.2014e-03, -4.2217e-02, -1.1914e-02, -2.9351e-02,
1.4702e-01, -2.4856e-02, -5.4605e-02, 9.2256e-02, -5.8185e-02,
-4.9112e-03, -4.8391e-02, 9.6750e-03, 9.3729e-02, -7.0812e-02,
1.3358e-02, -3.0346e-02, -1.2634e-02, 5.4181e-02, 3.5779e-03,
-1.3641e-02, 1.8990e-02, 4.1806e-02, 1.4102e-02, -3.5755e-02,
-9.7328e-03, -6.9314e-02, -1.1238e-02, 4.4573e-02, -5.3725e-02,
1.4389e-04, 3.9408e-02, -4.2209e-02, -3.2018e-02, 8.1474e-02,
9.0362e-03, 8.6104e-02, 1.3965e-01, -1.3486e-02, 3.1769e-02,
3.9424e-02, -2.7504e-02, 4.4009e-02, 4.5092e-02, 3.0902e-02,
9.5173e-02, 6.0828e-02, 1.8025e-03, -8.7061e-03, 6.5157e-03,
3.7598e-02, 1.0603e-01, 1.6571e-01, 5.5066e-02, -7.3351e-03,
-5.2073e-03, -6.4366e-02, -6.7670e-02, 5.9512e-02, 6.4599e-02,
-5.4156e-02, 3.0373e-02, -6.4649e-03, 6.1524e-02, -6.0863e-02,
-5.4839e-02, 3.2778e-02, -5.2311e-02, -6.3305e-02, -1.4740e-02,
8.6265e-03, -1.4593e-02, 7.0843e-02, 9.6598e-03, -4.1514e-02,
1.0386e-01, 9.7102e-02, 2.7971e-02, -8.3213e-03, 8.9160e-02,
6.4613e-02, 7.2493e-03, 2.0423e-02, -1.7861e-02, -1.0312e-02,
-2.7538e-02, -4.7007e-02, -5.1747e-02, 5.8215e-02, 6.6977e-02,
-1.7237e-02, -3.6396e-02, 5.7006e-02, 7.6054e-03, -4.7352e-02,
-9.1480e-03, 4.0137e-02, 6.6371e-03, 7.7091e-02, 5.0351e-02,
6.8603e-04, -5.7972e-02, 1.2907e-02, 3.9684e-02, -2.7078e-02,
5.0169e-02, 3.0860e-02, 5.1556e-02, -4.8597e-02, -2.1098e-02,
5.4887e-02, -4.1379e-02, 2.0082e-02, 1.8863e-02, -1.6039e-02,
4.4650e-02, -2.7449e-02, 2.2729e-02, 3.3439e-02, 6.7221e-02,
3.5019e-02, 1.2706e-02, -3.3244e-02, -2.0954e-02, -7.2121e-02,
-2.5076e-02, -9.3874e-02, -1.4208e-02, -4.0450e-02, 1.0401e-01,
-2.2949e-02, -2.6151e-02, 8.9983e-03, -7.2041e-02, -2.8341e-03,
-4.9221e-02, -4.2876e-02, 6.4064e-03, -2.5436e-02, 1.6128e-02,
-4.1882e-02, -3.8972e-02, -4.6335e-02, -5.4710e-02, -1.8748e-04,
-5.4616e-02, -5.6258e-02, -3.8972e-02, -2.6863e-02, 1.6232e-02,
1.6155e-02, -1.8721e-02, -2.4802e-02, -1.3311e-02, -1.7580e-02,
-4.3439e-02, -4.1209e-02, -2.6509e-02, 2.1775e-02, -1.2127e-02,
-1.7708e-02, -1.5500e-02, -7.4598e-02, -5.0580e-02, -6.6671e-02,
-4.6614e-02, 4.4161e-02, 1.9872e-02, 1.7880e-02, -7.2944e-03,
9.7257e-03, 1.2171e-01, 2.0895e-03, -7.2846e-02, -9.7906e-03,
-5.5273e-02, -1.8346e-03, 3.9121e-02, 1.7541e-02, 1.6898e-02,
2.0039e-02, -8.8893e-03, -2.4714e-02, -3.4171e-02, 1.0218e-02,
1.2038e-02, -5.3019e-02, 3.3046e-02, -3.6093e-02, -5.2178e-02,
-4.7642e-02, -6.6923e-02, -2.2623e-02, -7.2356e-02, -1.1350e-01,
-2.2522e-02, -1.8415e-02, -5.9977e-02, -2.2627e-02, 4.6349e-02],
requires_grad=True))]