pythonnumpypython-imaging-librarypython-mss

Accelerating a screenshot function - Python


I need my screenshot function to be as fast as possible, and now every call to the function takes about 0.2sec.

This is the function:

def get_screenshot(self, width, height):
    image = self.screen_capture.grab(self.monitor)
    image = Image.frombuffer('RGB', image.size, image.bgra, 'raw', 'BGRX')
    image = image.resize((int(width), int(height)), Image.BICUBIC) # Resize to the size of 0.8 from original picture
    image = np.array(image)
    image = np.swapaxes(image, 0, 1)
    # This code below supposed to replace each black color ([0,0,0]) to the color of [0,0,1]
    # r1,g1,b1 = [0,0,0] and r2,g2,b2 = [0,0,1]
    red, green, blue = image[:, :, 0], image[:, :, 1], image[:, :, 2]
    mask = (red == r1) & (green == g1) & (blue == b1)
    image[:, :, :3][mask] = [r2, g2, b2]
    return image

Do you notice any changes that I can do to make the function faster?

Edit: Some details that I forgot to mention:

  1. My screen dimensions are 1920*1080

  2. This function is a part of a live stream project that I am currently working on. The solution that Carlo has suggested below is not appropriate in this case because the remote computer will not be synchronized with our computer screen.


Solution

  • As your code is incomplete, I can only guess what might help, so here are a few thoughts...

    I started with a 1200x1200 image, because I don't know how big yours is, and reduced it by a factor of 0.8x to 960x960 because of a comment in your code.

    My ideas for speeding it up are based on either using a different interpolation method, or using OpenCV which is highly optimised SIMD code. Either, or both, may be appropriate, but as I don't know what your images look like, only you can say.

    So, here we go, first with PIL resize() and different interpolation methods:

    # Open image with PIL
    i = Image.open('start.png').convert('RGB')
    
    In [91]: %timeit s = i.resize((960,960), Image.BICUBIC)                                             
    16.2 ms ± 28 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
    
    In [92]: %timeit s = i.resize((960,960), Image.BILINEAR)                                            
    10.9 ms ± 87.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
    
    In [93]: %timeit s = i.resize((960,960), Image.NEAREST)                                             
    440 µs ± 10.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
    

    So, BILINEAR is 1.5x faster than BICUBIC and the real winner here is NEAREST at 32x faster.

    Now, converting to a Numpy array (as you are doing anyway) and using the highly optimised OpenCV SIMD code to resize:

    # Now make into Numpy array for OpenCV methods
    n = np.array(i)
    
    In [100]: %timeit s = cv2.resize(n, (960,960), interpolation = cv2.INTER_CUBIC)                     
    806 µs ± 9.81 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
    
    In [101]: %timeit s = cv2.resize(n, (960,960), interpolation = cv2.INTER_LINEAR)                    
    3.69 ms ± 29 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
    
    In [102]: %timeit s = cv2.resize(n, (960,960), interpolation = cv2.INTER_AREA)                      
    12.3 ms ± 136 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
    
    In [103]: %timeit s = cv2.resize(n, (960,960), interpolation = cv2.INTER_NEAREST)                   
    692 µs ± 448 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)
    

    And the winner here looks like INTER_CUBIC which is 20x faster than PIL's resize().

    Please try them all and see what works for you! Just remove the Python magic %timeit at the start of the line and run what's left.