pythonasynchronousaiohttp

Keep getting Asyncio/Aiohttp RuntimeError after code execution


This is part of the code section. I removed all useless stuff... I found out that the Player_Season variable causes this pipeline error. I'm not sure why its causing it. Hasn't been before.

Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001A221370AF0>
Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python39_64\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python39_64\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python39_64\lib\asyncio\base_events.py", line 751, in call_soon
    self._check_closed()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python39_64\lib\asyncio\base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
import asyncio
import aiohttp

class NHLTeam:
    async def fetch(self, session, url):
        headers = {
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Windows; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
        }
        async with session.get(url, headers=headers) as response:
            response.raise_for_status()  # Catch HTTP errors
            content = await response.json()
            return content

    async def scrape(self, data):
        print(data)
        async with aiohttp.ClientSession() as session:
            tasks = []
            for player_name, player_ID, year, position in data: #Iterate over each player
                Game_Type = 2 # 2 = Regular Season | 3 = Play offs
                Link = f"https://api-web.nhle.com/v1/player/{player_ID}/game-log/{year}/{Game_Type}"
                tasks.append(self.fetch_and_parse(session, player_name, Link, year, position))
            results = await asyncio.gather(*tasks)
            return results

    async def fetch_and_parse(self, session, player_name, player_url, year, position):
        Player_Season = await self.fetch(session, player_url) #Causes a Pipeline Error
        

team_1_organized_players_data = [['Connor',8483733, 20232024, 'Center'],['Connor',8483733, 20232024, 'Center']]
team_1 = NHLTeam()

async def run_scraping():
    await asyncio.gather(
        team_1.scrape(team_1_organized_players_data)
        #team_2.scrape(team_2_organized_players_data)
    )

asyncio.run(run_scraping())   

You can copy the code. It should work. But again I'm not sure why I keep getting the error.


Solution

  • Try to change the way you start the program using asyncio.run:

    if __name__ == "__main__":
        asyncio.set_event_loop(asyncio.ProactorEventLoop())
        asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
        team_1_organized_players_data = [['Connor',8483733, 20232024, 'Center'],['Connor',8483733, 20232024, 'Center']]
        team_1 = NHLTeam()
        asyncio.run(run_scraping())
    

    This should tell Python to use event loops that are designed for Windows.

    ProactorEventLoop() WindowsProactorEventLoopPolicy()