The library doesn't need to integrate with magento, it's mostly a wrapper that communicates with an API.
I would like to be able to use this library and make these API calls from within a controller or model.
Where can I put the library? How do I add them to the autoloader?
Look into /lib folder in your website root directory. From Magento Base Directories:
Magento’s library folder is where non-module based Magento code lives. This include a large amount of the system code which allows Magento to run, as well as a number of third party libraries (including the Zend Framework). The library is also the last code pool Magento will search when attempting to autoload a file.
So, in other words, if your library supports zend file naming convention - library classes will be found and loaded by magento autoloader. Otherwise you can get path of your /lib directory with Mage::getBaseDir(‘lib’) and write something like
require_once(Mage::getBaseDir('lib') . '/EZComponents/Base/src/base.php');