I have seen this similar question here, but not quite the same. I'm trying to get into a VM but with a Windows machine.
Any idea about what the problem could be?
I will assume that you are using PuTTY like ssh client in windows. You can download it from www.putty.org.
Putty does not allow to use private key file with .pem format. You should previously translate it into a .ppk file format.
Fortunately, PuTTY allows you translating this file using the PuTTYgen program (you can download it from www.putty.org as well), execute it and import the private key. If you see a successfully imported message you inly need to save the content of the .ppk file into your system and work with it in PuTTY (select the private key file for authentication).
I hope that this resolve the problem.