pythonweb-scraping

Using Python to Download CVS "Download CSV" Button

I'm trying to download the csv file using Python from this site:https://gats.pjm-eis.com/gats2/PublicReports/GATSGenerators

There's a csv button in the top right corner that I want to automatically load into a data warehouse. I've gone through a few tutorials (new to Python) and have yet to be successful. Any recommendations?

Solution

  • Use the library called requests, it is:

    import requests

    You need it to create the request to the cvs resource.

    Also there's a library used for screen-scraping called bs4

    import bs4

    You will need both to construct what you want. Look for a course over there on web scraping with python and bs4.

    Also there's a library called csv, 

    import csv

    You can use it to easily parse the csv file once you get it.

    Check this example or google it:

    https://www.digitalocean.com/community/tutorials/how-to-scrape-web-pages-with-beautiful-soup-and-python-3

    Here's another course on LinkedIn learning platform

    https://www.linkedin.com/learning/scripting-for-testers/welcome