ssh-add -l displays that I have 3 RSA keys added to my SSH agent.
ssh-add -L displays those public keys.
How do I get the private keys as well, so that I can save them to a file? Or is it by design that this is impossible? How does it work then?
Can ssh-agent be asked to do operations using the private key? How can I ask it to encrypt/decrypt a number for me?
It's OK that I have to write code for this (the programming language doesn't matter), but I'd prefer using an existing tool or a library.
It's not possible to get the private key or to perform encryption using the protocol between ssh and ssh-agent, but it's possible to get the private key by dumping the memory of the ssh-agent. On Linux you have to be root to do the memory dump.
I've just found a very good explanation about how ssh-agent works: http://www.unixwiz.net/techtips/ssh-agent-forwarding.html . This partially answers some of my questions.
One of the more clever aspects of the agent is how it can verify a user's identity (or more precisely, possession of a private key) without revealing that private key to anybody.
One of the security benefits of agent forwarding is that the user's private key never appears on remote systems or on the wire, even in encrypted form.
Thus the protocol between the SSH client and the ssh-agent proviedes no way in SSH1 or SSH2 to get out the private keys from an ssh-agent.
However, as root you can get a memory dump of ssh-agent, and try to extract the private key from there. https://blog.netspi.com/stealing-unencrypted-ssh-agent-keys-from-memory does exactly that, and there are other pieces of software mentioned in the comment section of that page. However, the software on that page didn't work for me on Debian buster: the memory dump didn't contain any keys, even though ssh-add -l has displayed an RSA key.