pythonauthenticationpython-requestsaiohttp.netrc

Make .netrc authentication work with aiohttp


I'd like to use a .netrc file with credentials to authenticate to an API using aiohttp. As far as I can tell this should be possible, as long as the file is in the home directory (or the relevant env variable set correctly) and trust_env=True set in the aiohttp.ClientSession.

But whatever I try, I get a 401 response. I've checked with requests, and it works just fine. I've browsed through the relevant code and it seems like it'll only pick up the credentials if a proxy is supplied. Can someone explain?

Here's an example that reproduces the issue:

First put a .netrc file in home directory:

machine httpbin.org
    login foo
    password bar
import aiohttp
import requests

url = "http://httpbin.org/basic-auth/foo/bar"

with requests.Session() as sess:
    r = sess.get(url)
    r.raise_for_status()

# no exception raised

async with aiohttp.ClientSession(trust_env=True) as session:
    r = await session.get(url)
    r.raise_for_status()

# exception raised

ClientResponseError: 401, message='UNAUTHORIZED', url=URL('http://httpbin.org/basic-auth/foo/bar')


Solution

  • From what I understand in the doc, the trust_env and .netrc credentials are only used for proxy authentication, not for regular server authentication.

    For authentication to the server directly, the docs say that you have to use a BasicAuth object (as you surely know), but to use the .netrc file, one solution would be to use a custom authentication class, eg:

    class NetrcAuth(aiohttp.BasicAuth):
        def __new__(cls, host):
            login, account, password = netrc.netrc().authenticators(host)
            return super().__new__(cls, login=login, password=password)
    

    that you could then use as

    from urllib.parse import urlparse
    
    hostname = urlparse(url).hostname
    
    async with aiohttp.ClientSession(auth=NetrcAuth(hostname)) as session:
        r = await session.get(url)
        r.raise_for_status()
    

    Of course this is not optimal, as we would like to have the ClientSession take care of that for us, but it's maybe a step in the right direction?