Tensorflow, OpenAI Gym, Keras-rl performance issue on basic reinforcement learning example

I'm doing reinforcement learning, and I'm having trouble with performance.

Situation, no custom code:

• I loaded a Google Deep Learning VM (https://console.cloud.google.com/marketplace/details/click-to-deploy-images/deeplearning) on Google Cloud. This comes with all the prerequisites installed (CUDA, cuDNN, drivers) with a NVidia K80 videocard.
• Installed keras-rl, OpenAI gym
• Now when I run the (standard) example dqn_cartpole.py with visualize=False on line 46, it uses about 20% of my GPU, resulting in around 100 steps per second, which is around 3x SLOWER than using the CPU on my Razer Blade 15 (i7-8750H).
• I have checked all the bottlenecks I can think of, CPU usage, memory and HD I/O is all normal.

Please help!

Thanks in advance

This is not necessarily a problem. Using a GPU does not come "for free" in terms of performance, and it's not always faster than a CPU. Because not everything runs on GPU (e.g. the gym environment itself still runs on CPU), you do incur "communication costs" (e.g. moving memory to and from the GPU). This will only be worth it if you can really make good use of your GPU.

Now, GPUs are also not necessarily faster than CPUs. GPUs are very good at performing lots of similar computations in parallel. This is necessary, for example, for matrix multiplications between large matrices, which indeed happens fairly often when training large, deep Neural Networks. If you only need a relatively small number of computations that can be done in parallel like that, and mostly just have sequential code, a GPU definitely can be slower than a CPU (and the CPU you mentioned is a rather powerful one).

Now, if I look at the part of the code you linked where the Neural Network is being built (starting from line 22), that looks like a rather small Neural Network; just a few layers of 16 nodes each. This is not a huge Neural Network, with Convolutional layers followed up by large (e.g. hundreds of nodes) fully connected layers (which would likely indeed be overkill for a small problem like Cartpole). So, it's certainly no surprise that you only get to use 20% of the GPU (it simply can't use more in parallel because the matrices being multiplied are too small), and not necessarily a surprise that it'd be slower than simply running on a CPU either.