[SOLVED] The library libcrypto could not be found

Recently my lambda code stopped working. I am no longer able to create connection to Snowflake with sqlalchemy. See error stack below.

The library libcrypto could not be found: LibraryNotFoundError
Traceback (most recent call last):
  (...)
  File "/var/task/sqlalchemy/engine/base.py", line 2263, in connect
    return self._connection_cls(self, **kwargs)
  File "/var/task/sqlalchemy/engine/base.py", line 104, in __init__
    else engine.raw_connection()
  File "/var/task/sqlalchemy/engine/base.py", line 2370, in raw_connection
    self.pool.unique_connection, _connection
  File "/var/task/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect
    return fn()
  File "/var/task/sqlalchemy/pool/base.py", line 304, in unique_connection
    return _ConnectionFairy._checkout(self)
  File "/var/task/sqlalchemy/pool/base.py", line 778, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/var/task/sqlalchemy/pool/base.py", line 495, in checkout
    rec = pool._do_get()
  File "/var/task/sqlalchemy/pool/impl.py", line 140, in _do_get
    self._dec_overflow()
  File "/var/task/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    with_traceback=exc_tb,
  File "/var/task/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/var/task/sqlalchemy/pool/impl.py", line 137, in _do_get
    return self._create_connection()
  File "/var/task/sqlalchemy/pool/base.py", line 309, in _create_connection
    return _ConnectionRecord(self)
  File "/var/task/sqlalchemy/pool/base.py", line 440, in __init__
    self.__connect(first_connect_check=True)
  File "/var/task/sqlalchemy/pool/base.py", line 661, in __connect
    pool.logger.debug("Error on connect(): %s", e)
  File "/var/task/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    with_traceback=exc_tb,
  File "/var/task/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/var/task/sqlalchemy/pool/base.py", line 656, in __connect
    connection = pool._invoke_creator(self)
  File "/var/task/sqlalchemy/engine/strategies.py", line 114, in connect
    return dialect.connect(*cargs, **cparams)
  File "/var/task/sqlalchemy/engine/default.py", line 509, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/var/task/snowflake/connector/__init__.py", line 52, in Connect
    return SnowflakeConnection(**kwargs)
  File "/var/task/snowflake/connector/connection.py", line 227, in __init__
    self.connect(**kwargs)
  File "/var/task/snowflake/connector/connection.py", line 423, in connect
    self.__open_connection()
  File "/var/task/snowflake/connector/connection.py", line 633, in __open_connection
    self._authenticate(auth_instance)
  File "/var/task/snowflake/connector/connection.py", line 866, in _authenticate
    self.__authenticate(self.__preprocess_auth_instance(auth_instance))
  File "/var/task/snowflake/connector/connection.py", line 896, in __authenticate
    session_parameters=self._session_parameters,
  File "/var/task/snowflake/connector/auth.py", line 200, in authenticate
    socket_timeout=self._rest._connection.login_timeout)
  File "/var/task/snowflake/connector/network.py", line 518, in _post_request
    _include_retry_params=_include_retry_params)
  File "/var/task/snowflake/connector/network.py", line 595, in fetch
    **kwargs)
  File "/var/task/snowflake/connector/network.py", line 702, in _request_exec_wrapper
    raise e
  File "/var/task/snowflake/connector/network.py", line 633, in _request_exec_wrapper
    **kwargs)
  File "/var/task/snowflake/connector/network.py", line 891, in _request_exec
    raise err
  File "/var/task/snowflake/connector/network.py", line 787, in _request_exec
    auth=SnowflakeAuth(token),
  File "/var/task/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/var/task/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/var/task/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/var/task/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/var/task/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/var/task/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/var/task/urllib3/connection.py", line 371, in connect
    ssl_context=context,
  File "/var/task/snowflake/connector/ssl_wrap_socket.py", line 74, in ssl_wrap_socket_with_ocsp
    from .ocsp_asn1crypto import SnowflakeOCSPAsn1Crypto as SFOCSP
  File "/var/task/snowflake/connector/ocsp_asn1crypto.py", line 48, in <module>
    from oscrypto import asymmetric
  File "/var/task/oscrypto/asymmetric.py", line 19, in <module>
    from ._asymmetric import _unwrap_private_key_info
  File "/var/task/oscrypto/_asymmetric.py", line 27, in <module>
    from .kdf import pbkdf1, pbkdf2, pkcs12_kdf
  File "/var/task/oscrypto/kdf.py", line 9, in <module>
    from .util import rand_bytes
  File "/var/task/oscrypto/util.py", line 14, in <module>
    from ._openssl.util import rand_bytes
  File "/var/task/oscrypto/_openssl/util.py", line 6, in <module>
    from ._libcrypto import libcrypto, libcrypto_version_info, handle_openssl_error
  File "/var/task/oscrypto/_openssl/_libcrypto.py", line 9, in <module>
    from ._libcrypto_cffi import (
  File "/var/task/oscrypto/_openssl/_libcrypto_cffi.py", line 27, in <module>
    raise LibraryNotFoundError('The library libcrypto could not be found')
oscrypto.errors.LibraryNotFoundError: The library libcrypto could not be found

The library libcrypto could not be found: LibraryNotFoundError Traceback (most recent call last): File "/var/task/index.py", line 43, in lambda_handler main(event['survey_id'], event['k_anon_lvl'], sf_config_imm_deployer, immuta_config, sf_config_immuta) File "/var/task/immuta_bruteforce_table.py", line 442, in main p = Project(immuta_config, sf_config, project_name, delete_existing=True) File "/var/task/immuta_bruteforce_table.py", line 212, in __init__ self.delete_schema() File "/var/task/immuta_bruteforce_table.py", line 253, in delete_schema self.sf_config.connect() File "/var/task/immuta_bruteforce_table.py", line 109, in connect self.connection = self.engine.connect() File "/var/task/sqlalchemy/engine/base.py", line 2263, in connect return self._connection_cls(self, **kwargs) File "/var/task/sqlalchemy/engine/base.py", line 104, in __init__ else engine.raw_connection() File "/var/task/sqlalchemy/engine/base.py", line 2370, in raw_connection self.pool.unique_connection, _connection File "/var/task/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect return fn() File "/var/task/sqlalchemy/pool/base.py", line 304, in unique_connection return _ConnectionFairy._checkout(self) File "/var/task/sqlalchemy/pool/base.py", line 778, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/var/task/sqlalchemy/pool/base.py", line 495, in checkout rec = pool._do_get() File "/var/task/sqlalchemy/pool/impl.py", line 140, in _do_get self._dec_overflow() File "/var/task/sqlalchemy/util/langhelpers.py", line 70, in __exit__ with_traceback=exc_tb, File "/var/task/sqlalchemy/util/compat.py", line 182, in raise_ raise exception File "/var/task/sqlalchemy/pool/impl.py", line 137, in _do_get return self._create_connection() File "/var/task/sqlalchemy/pool/base.py", line 309, in _create_connection return _ConnectionRecord(self) File "/var/task/sqlalchemy/pool/base.py", line 440, in __init__ self.__connect(first_connect_check=True) File "/var/task/sqlalchemy/pool/base.py", line 661, in __connect pool.logger.debug("Error on connect(): %s", e) File "/var/task/sqlalchemy/util/langhelpers.py", line 70, in __exit__ with_traceback=exc_tb, File "/var/task/sqlalchemy/util/compat.py", line 182, in raise_ raise exception File "/var/task/sqlalchemy/pool/base.py", line 656, in __connect connection = pool._invoke_creator(self) File "/var/task/sqlalchemy/engine/strategies.py", line 114, in connect return dialect.connect(*cargs, **cparams) File "/var/task/sqlalchemy/engine/default.py", line 509, in connect return self.dbapi.connect(*cargs, **cparams) File "/var/task/snowflake/connector/__init__.py", line 52, in Connect return SnowflakeConnection(**kwargs) File "/var/task/snowflake/connector/connection.py", line 227, in __init__ self.connect(**kwargs) File "/var/task/snowflake/connector/connection.py", line 423, in connect self.__open_connection() File "/var/task/snowflake/connector/connection.py", line 633, in __open_connection self._authenticate(auth_instance) File "/var/task/snowflake/connector/connection.py", line 866, in _authenticate self.__authenticate(self.__preprocess_auth_instance(auth_instance)) File "/var/task/snowflake/connector/connection.py", line 896, in __authenticate session_parameters=self._session_parameters, File "/var/task/snowflake/connector/auth.py", line 200, in authenticate socket_timeout=self._rest._connection.login_timeout) File "/var/task/snowflake/connector/network.py", line 518, in _post_request _include_retry_params=_include_retry_params) File "/var/task/snowflake/connector/network.py", line 595, in fetch **kwargs) File "/var/task/snowflake/connector/network.py", line 702, in _request_exec_wrapper raise e File "/var/task/snowflake/connector/network.py", line 633, in _request_exec_wrapper **kwargs) File "/var/task/snowflake/connector/network.py", line 891, in _request_exec raise err File "/var/task/snowflake/connector/network.py", line 787, in _request_exec auth=SnowflakeAuth(token), File "/var/task/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/var/task/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/var/task/requests/adapters.py", line 449, in send timeout=timeout File "/var/task/urllib3/connectionpool.py", line 677, in urlopen chunked=chunked, File "/var/task/urllib3/connectionpool.py", line 381, in _make_request self._validate_conn(conn) File "/var/task/urllib3/connectionpool.py", line 978, in _validate_conn conn.connect() File "/var/task/urllib3/connection.py", line 371, in connect ssl_context=context, File "/var/task/snowflake/connector/ssl_wrap_socket.py", line 74, in ssl_wrap_socket_with_ocsp from .ocsp_asn1crypto import SnowflakeOCSPAsn1Crypto as SFOCSP File "/var/task/snowflake/connector/ocsp_asn1crypto.py", line 48, in <module> from oscrypto import asymmetric File "/var/task/oscrypto/asymmetric.py", line 19, in <module> from ._asymmetric import _unwrap_private_key_info File "/var/task/oscrypto/_asymmetric.py", line 27, in <module> from .kdf import pbkdf1, pbkdf2, pkcs12_kdf File "/var/task/oscrypto/kdf.py", line 9, in <module> from .util import rand_bytes File "/var/task/oscrypto/util.py", line 14, in <module> from ._openssl.util import rand_bytes File "/var/task/oscrypto/_openssl/util.py", line 6, in <module> from ._libcrypto import libcrypto, libcrypto_version_info, handle_openssl_error File "/var/task/oscrypto/_openssl/_libcrypto.py", line 9, in <module> from ._libcrypto_cffi import ( File "/var/task/oscrypto/_openssl/_libcrypto_cffi.py", line 27, in <module> raise LibraryNotFoundError('The library libcrypto could not be found') oscrypto.errors.LibraryNotFoundError: The library libcrypto could not be found

I checked and libcrypto package is installed via PIP.

2021-01-12T21:09:03.711+01:00   Python version

2021-01-12T21:09:03.711+01:00   3.6.12 (default, Nov 4 2020, 20:34:46)

2021-01-12T21:09:03.711+01:00   [GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]

2021-01-12T21:09:03.711+01:00   Version info.

2021-01-12T21:09:03.711+01:00   sys.version_info(major=3, minor=6, micro=12, releaselevel='final', serial=0)

2021-01-12T21:09:03.711+01:00   platform

2021-01-12T21:09:03.719+01:00   Linux-4.14.203-112.332.amzn2.x86_64-x86_64-with-glibc2.2.5

2021-01-12T21:09:03.719+01:00   PIP

2021-01-12T21:09:03.802+01:00   Package Version

2021-01-12T21:09:03.802+01:00   -------------------------- ---------

2021-01-12T21:09:03.858+01:00   asn1crypto 1.4.0

2021-01-12T21:09:03.859+01:00   azure-common 1.1.26

2021-01-12T21:09:03.859+01:00   azure-core 1.10.0

2021-01-12T21:09:03.859+01:00   azure-storage-blob 12.6.0

2021-01-12T21:09:03.859+01:00   boto3 1.16.52

2021-01-12T21:09:03.859+01:00   botocore 1.19.52

2021-01-12T21:09:03.859+01:00   certifi 2020.12.5

2021-01-12T21:09:03.859+01:00   cffi 1.14.4

2021-01-12T21:09:03.859+01:00   chardet 3.0.4

2021-01-12T21:09:03.859+01:00   cryptography 3.3.1

2021-01-12T21:09:03.859+01:00   idna 2.10

2021-01-12T21:09:03.859+01:00   isodate 0.6.0

2021-01-12T21:09:03.859+01:00   jmespath 0.10.0

2021-01-12T21:09:03.859+01:00   msrest 0.6.19

2021-01-12T21:09:03.859+01:00   numpy 1.19.5

2021-01-12T21:09:03.859+01:00   oauthlib 3.1.0

2021-01-12T21:09:03.859+01:00   oscrypto 1.2.1

2021-01-12T21:09:03.859+01:00   pandas 0.24.1

2021-01-12T21:09:03.859+01:00   pip 18.1

2021-01-12T21:09:03.859+01:00   pycparser 2.20

2021-01-12T21:09:03.859+01:00   pycrypto 2.6.1

2021-01-12T21:09:03.860+01:00   pycryptodomex 3.9.9

2021-01-12T21:09:03.860+01:00   PyJWT 1.7.1

2021-01-12T21:09:03.860+01:00   pyOpenSSL 20.0.1

2021-01-12T21:09:03.860+01:00   python-dateutil 2.8.1

2021-01-12T21:09:03.860+01:00   pytz 2020.5

2021-01-12T21:09:03.860+01:00   requests 2.23.0

2021-01-12T21:09:03.860+01:00   requests-oauthlib 1.3.0

2021-01-12T21:09:03.860+01:00   s3transfer 0.3.4

2021-01-12T21:09:03.860+01:00   setuptools 40.6.2

2021-01-12T21:09:03.860+01:00   six 1.15.0

2021-01-12T21:09:03.860+01:00   snowflake-connector-python 2.3.7

2021-01-12T21:09:03.860+01:00   snowflake-sqlalchemy 1.2.0

2021-01-12T21:09:03.860+01:00   SQLAlchemy 1.3.22

2021-01-12T21:09:03.860+01:00   urllib3 1.25.11

2021-01-12T21:09:04.140+01:00   The directory '/home/sbx_user1051/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

2021-01-12T21:09:06.059+01:00   You are using pip version 18.1, however version 20.3.3 is available.

2021-01-12T21:09:06.059+01:00   You should consider upgrading via the 'pip install --upgrade pip' command.

Have you ever experienced similar problem? Please advice how to debug and where to look for problem details.