pythongoogle-cloud-functionsgspreadgoogle-secret-manager

In Google Cloud Function, gspread_pandas gives TypeError: 'AuthorizedSession' object is not callable. Loading credentials from Secret Manager


I'm trying to deploy a Google Cloud Function (v1) using Python 3.12. I'm loading the credentials for gspread_pandas as a json string from Google Secrets Manager into an environment variable. The code is as follows; I'm trying to remove everything that isn't relevant.

import json
import gspread_pandas as gsp

credentials_json = env_var["SHEETS_CREDENTIALS"]
credentials_sheets = json.loads(credentials_json)
gsFileKey = env_var['FILE_KEY']


scope = ['https://spreadsheets.google.com/feeds','https://www.googleapis.com/auth/drive']
targetFile = gsp.Spread(gsFileKey, config=credentials_sheets)

The trace from the deployment command:

Deploying function (may take a while - up to 2 minutes)...failed.
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Function failed on loading user code. This is likely due to a bug in the user code. Error message:     return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/functions_framework/_cli.py", line 37, in _cli
    app = create_app(target, source, signature_type)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/functions_framework/__init__.py", line 288, in create_app
    spec.loader.exec_module(source_module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/workspace/main.py", line 49, in <module>
    targetFile = gsp.Spread(gsFileKey, config=credentials_sheets)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/gspread_pandas/spread.py", line 129, in __init__
    self.client = Client(user, config, scope, creds)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/gspread_pandas/client.py", line 92, in __init__
    super().__init__(credentials, session)
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/gspread/client.py", line 41, in __init__
    self.http_client = http_client(auth)
                       ^^^^^^^^^^^^^^^^^
TypeError: 'AuthorizedSession' object is not callable.

The secret, when viewed in Secret Manager, looks like this (it's one long line): {"type": "service_account","project_id":<..stuff I cut out for obvious reasons...>m.gserviceaccount.com"}

For completeness sake, the output of conda list for this environment:

# Name                    Version                   Build  Channel   
abseil-cpp                20230802.0           h5da7b33_2
aiohttp                   3.9.0           py312h2bbff1b_0
aiosignal                 1.2.0              pyhd3eb1b0_0
asn1crypto                1.5.1                    pypi_0    pypi    
attrs                     23.1.0          py312haa95532_0
blinker                   1.6.2           py312haa95532_0
brotli-python             1.0.9           py312hd77b12b_7
bzip2                     1.0.8                he774522_0
c-ares                    1.19.1               h2bbff1b_0
ca-certificates           2023.12.12           haa95532_0
cachetools                4.2.2              pyhd3eb1b0_0
certifi                   2023.11.17      py312haa95532_0
cffi                      1.16.0          py312h2bbff1b_0
charset-normalizer        2.0.4              pyhd3eb1b0_0
click                     8.1.7           py312haa95532_0
cloud-sql-python-connector 1.6.0                    pypi_0    pypi
cloudevents               1.10.1                   pypi_0    pypi
colorama                  0.4.6           py312haa95532_0
cryptography              41.0.7          py312h89fc84f_0
decorator                 5.1.1                    pypi_0    pypi
deprecation               2.1.0                    pypi_0    pypi
expat                     2.5.0                hd77b12b_0
flask                     3.0.1                    pypi_0    pypi
frozenlist                1.4.0           py312h2bbff1b_0
functions-framework       3.5.0                    pypi_0    pypi
google-api-core           2.11.1             pyhd8ed1ab_0    conda-forge
google-api-core-grpc      2.11.1               hd8ed1ab_0    conda-forge
google-auth               2.27.0             pyhca7485f_0    conda-forge
google-auth-oauthlib      0.5.2           py312haa95532_0
google-cloud-secret-manager 2.18.0             pyhd8ed1ab_0    conda-forge
googleapis-common-protos  1.56.4          py312haa95532_0
googleapis-common-protos-grpc 1.56.4          py312haa95532_0
greenlet                  3.0.1           py312hd77b12b_0
grpc-cpp                  1.48.2               h6772dbd_4
grpc-google-iam-v1        0.13.0             pyhd8ed1ab_0    conda-forge
grpcio                    1.48.2          py312h6772dbd_4
grpcio-status             1.41.1             pyhd3eb1b0_0
gspread                   5.12.4             pyh1a96a4e_0    conda-forge
gspread-pandas            3.2.3                    pypi_0    pypi
gtest                     1.14.0               h59b6b97_0
idna                      3.4             py312haa95532_0
itsdangerous              2.1.2                    pypi_0    pypi
jinja2                    3.1.3                    pypi_0    pypi
libffi                    3.4.4                hd77b12b_0
libprotobuf               3.20.3               h23ce68f_0
markupsafe                2.1.4                    pypi_0    pypi
multidict                 6.0.4           py312h2bbff1b_0
numpy                     1.26.3                   pypi_0    pypi
oauthlib                  3.2.2           py312haa95532_0
openssl                   3.0.13               h2bbff1b_0
packaging                 23.2                     pypi_0    pypi
pandas                    2.2.0                    pypi_0    pypi
pg8000                    1.30.4                   pypi_0    pypi
pip                       23.3.1          py312haa95532_0
proto-plus                1.23.0             pyhd8ed1ab_0    conda-forge
protobuf                  3.20.3          py312hd77b12b_0
pyasn1                    0.4.8              pyhd3eb1b0_0
pyasn1-modules            0.2.8                      py_0
pycparser                 2.21               pyhd3eb1b0_0
pyjwt                     2.4.0           py312haa95532_0
pyopenssl                 23.2.0          py312haa95532_0
pysocks                   1.7.1           py312haa95532_0
python                    3.12.1               h1d929f7_0
python-dateutil           2.8.2                    pypi_0    pypi
pytz                      2023.4                   pypi_0    pypi
pyu2f                     0.1.5              pyhd8ed1ab_0    conda-forge
re2                       2022.04.01           hd77b12b_0
requests                  2.31.0          py312haa95532_0
requests-oauthlib         1.3.0                      py_0
rsa                       4.7.2              pyhd3eb1b0_1
scramp                    1.4.4                    pypi_0    pypi
setuptools                68.2.2          py312haa95532_0
six                       1.16.0             pyhd3eb1b0_1
sqlalchemy                2.0.25          py312h2bbff1b_0
sqlite                    3.41.2               h2bbff1b_0
tk                        8.6.12               h2bbff1b_0
typing-extensions         4.9.0           py312haa95532_1
typing_extensions         4.9.0           py312haa95532_1
tzdata                    2023.4                   pypi_0    pypi
urllib3                   1.26.18         py312haa95532_0
vc                        14.2                 h21ff451_1
vs2015_runtime            14.27.29016          h5e58377_2
watchdog                  3.0.0                    pypi_0    pypi
werkzeug                  3.0.1                    pypi_0    pypi
wheel                     0.41.2          py312haa95532_0
win_inet_pton             1.1.0           py312haa95532_0
xz                        5.4.5                h8cc25b3_0
yarl                      1.9.3           py312h2bbff1b_0
zlib                      1.2.13               h8cc25b3_0

I ran the same function locally using functions-framework with the environment variable loaded in manually, which works. I've also tried accessing the credentials using the Python google.cloud.secretmanager library instead, which also works running locally in functions-framework but not in the Cloud. The same code does work with the credentials hard coded into the script, but my whole reason for rewriting it was to get rid of the hard coded credentials. I've used this combination of cloud function, service account, secret (through the library) and gspread (non-pandas) in a different Cloud function before without issue (Python 3.10). I can't really think of what else could be wrong.


Solution

  • I had the same issue. gspread-pandas had updated the gspread package to version 6.0.0. By reverting to version gspread==5.12.4, it now works. In version 5.12.4, it accepts "auth" and "session" as parameters, which are the ones passed by gspread_pandas. However, in version 6.0.0 (and also in 6.0.1, which is partially fixed) the parameters are different.