I created a droplet on digitalocean, created a sudo-user and made ssh provision to access the droplet from my VScode temrminal. On trying to access from the terminal, I got a dialog prompting me to "Enter password to unlock the private key".
Turned out that while generating the ssh key, the passphrase that was entered, was exactly the password being requested.