Getting xvfb to work in jupyter notebook on M1 Mac

I am working with OpenAI gym on colab and using the following code to render the videos within Jupyter notebook based on this tutorial: https://colab.research.google.com/drive/1flu31ulJlgiRL1dnN2ir8wGh9p7Zij2t

import gym
from gym import logger as gymlogger
from gym.wrappers import Monitor
gymlogger.set_level(40) #error only
import numpy as np
import random
import matplotlib
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
import math
import glob
import io
import base64

from IPython.display import HTML
from IPython import display as ipythondisplay, display

import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, ConvLSTM2D, Flatten, Conv2D
from tf_agents.agents.dqn import dqn_agent
from tf_agents.replay_buffers import tf_uniform_replay_buffer
from tf_agents.specs import tensor_spec

display = Display(visible=0, size=(1400, 900))
display.start()

def show_video():
  mp4list = glob.glob('video/*.mp4')
  if len(mp4list) > 0:
    mp4 = mp4list[0]
    video = io.open(mp4, 'r+b').read()
    encoded = base64.b64encode(video)
    ipythondisplay.display(HTML(data='''<video alt='test' autoplay 
                loop controls style='height: 400px;'>
                <source src='data:video/mp4;base64,{0}' type='video/mp4' />
             </video>'''.format(encoded.decode('ascii'))))
  else: 
    print('Could not find video')
    

def wrap_env(env):
  env = Monitor(env, './video', force=True)
  return env

env, spec = query_environment('MsPacman-v0')
env = wrap_env(env)

observation = env.reset()

while True:
    env.render()
    #your agent goes here
    action = env.action_space.sample() 
    observation, reward, done, info = env.step(action) 
    if done: 
        break;
            
env.close()
show_video()

On my M1 Mac, I cannot run set up the display with these two lines below due to missing xvfb. All my packages on the M1 Mac are running natively.

display = Display(visible=0, size=(1400, 900))
display.start()

I have tried installing xvfbwrapper and importing Xvfb then and replacing the first line as shown below but the error is can not find xvfb. I read and found that xvfb is not shipped with Macs any more so I installed it via Xquartz: https://www.xquartz.org/ but it still does not work.

from xvfbwrapper import Xvfb
display = Xvfb(width=1400, height=900)

Without these lines, there is still a video generated but it is in a new window and not within jupyter notebook itself. (There is also an issue that I cannot seem to close that window once it is generated but that is another issue).

My question is whether there is any modification I can do to make the code above work? If not, are there any other ways to render the video within the notebook itself?

Thank you.

Not sure why this is the case even though xvfb is in the PATH but had to add it again specifically to the environment in a jupyter notebook cell for it to work

os.environ["PATH"] += os.pathsep + '/opt/X11/bin'