Setting the luminance of an image in skimage does not return constant luminance

I have an rgb image (rgb), and I want to set the luminance to make sure that it's constant. I first check the luminance of the original image (greyrgb). I then convert the image to Lab, then setting the L value to 50 (lab50), before converting back to rgb (rgb50). I then compute the luminance of the resulting image (greyrgb50), but it is not constant - it actually seems worse.

import numpy as np
from skimage import color
import matplotlib.pyplot as plt

rgb = np.loadtxt("my_image.txt").reshape((512,512,4))
greyrgb = color.rgb2gray(rgb)

lab = color.rgb2lab(rgb)
lab50 = lab
lab50[:,:,0] = 50
rgb50 = color.lab2rgb(lab50)
greyrgb50 = color.rgb2gray(rgb50)

I find that when I set the luminance to different values, the luminance of the resultant rgb image still varies slightly. Have I set the luminance incorrectly, or am I computing the luminance of the final rgb image incorrectly?

You are trying to create out of gamut colors. The colors, that can be written as numbers, but cannot exist in the real life or other color spaces. For example, maximum luminance of the pure red color cannot exceed 0.3, maximum luminance of the pure blue color cannot exceed 0.1, green is more or less fine, but still cannot exceeed 0.6 -- hence you have darker areas where only one color of (r,g,b) is present, because it cannot be correctly represented in RGB color space.