Sharepoint Rest API office365 library 403 Client Error

I am trying to access a Sharepoint folder and retrieve the files within.

With the following code:

# Import libraries
from office365.sharepoint.client_context import ClientContext
from office365.runtime.auth.client_credential import ClientCredential

# Config for Sharepoint (to be stored in env later)
SP_CLIENT_ID = xxx
SP_CLIENT_SECRET = xxx
SP_URL = 'https://<organization_url>.sharepoint.com/sites/<site-name>'
relative_folder_url = '/Shared%20Documents/<subfolder>'

# App-based authentication with access credentials
context = ClientContext(SP_URL).with_credentials(ClientCredential(SP_CLIENT_ID, SP_CLIENT_SECRET))
folder = context.web.get_folder_by_server_relative_url(relative_folder_url)
context.load(folder)
context.execute_query()

# Processes each Sharepoint file in folder
for file in folder.files:
    print(f'Processing file: {file.properties["Name"]}')

However, it throws an error

office365.runtime.client_request_exception.ClientRequestException: ('-2147024891, System.UnauthorizedAccessException', 'Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))', "403 Client Error: Forbidden for url: https://<tenant-name>.sharepoint.com/sites/<site-id>/_api/Web/getFileByServerRelativePath(DecodedUrl='%2Fsites%2F<site-name>%2FDocuments%2FGeneral%2F<subfolder>')")` stemming from `context.execute_query()

with the specific names replaced by <xxx>.

I am able to retrieve files using Graph API on Postman, using the same SP_CLIENT_ID and SP_CLIENT_SECRET. It is done by generating a token using the client ID and secret, then passing token in Graph GET API calls.

It appears the app-based authentication is fine as the error is 403.

How can it be fixed such that files can be programmatically seen (and ultimately downloaded) through Sharepoint Rest API on Python?

Edit: The error is likely coming from the lack of permission granted for Sharepoint Rest API on the Azure app I am using to authenticate with Sharepoint. Now I am thinking if I should continue using office365 or simply use some pure API approach like requests library in Python to programmatically download Sharepoint files.

I have one SharePoint site named sritestsite10 having few files in following path:

When I ran your code in my environment, I too got same error as below:

To resolve the error, I ran below URL with Global Administrator account and added app-only principal with tenant permissions like this:

https://xxx-admin.sharepoint.com/_layouts/15/appinv.aspx

You can find your Azure AD application by entering AppId and use below xml for adding permissions:

<AppPermissionRequests AllowAppOnlyPolicy="true">
  <AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" />
</AppPermissionRequests>

Make sure to click on Trust it option to grant the permissions:

When I ran the code again by adding .expand(["Files"]) to folder variable, I got the response successfully as below:

# Import libraries
from office365.sharepoint.client_context import ClientContext
from office365.runtime.auth.client_credential import ClientCredential

# Config for Sharepoint (to be stored in env later)
SP_CLIENT_ID = xxx
SP_CLIENT_SECRET = xxx
SP_URL = 'https://<organization_url>.sharepoint.com/sites/<site-name>'
relative_folder_url = '/Shared%20Documents/<subfolder>'

# App-based authentication with access credentials
context = ClientContext(SP_URL).with_credentials(ClientCredential(SP_CLIENT_ID, SP_CLIENT_SECRET))
folder = context.web.get_folder_by_server_relative_url(relative_folder_url).expand(["Files"])
context.load(folder)
context.execute_query()

# Processes each Sharepoint file in folder
for file in folder.files:
    print(f'Processing file: {file.properties["Name"]}')

Response:

Reference: azure - HTTPError: 403 Client Error: Forbidden for url via Office365-REST-Python-Client - Stack Overflow by Rukmini