pythonpython-3.xaws-lambda

Unable to use python library - "smbprotocol" (1.9.0) in AWS Lambda

I want to use the smbprotocol python library in my Lambda function.

I tried adding it as a deployment package:

mypackage.zip
|
|--> packages
|    |
|    |-->pip
|    |-->smbprotocol
|    |-->..
|--> myscript.py

Gave me the below error:

[ERROR] Runtime.ImportModuleError: Unable to import module 'generate_sequenceFile_submit_request': cannot import name 'x509' from 'cryptography.hazmat.bindings._rust'

I tried creating a Layer and using it through layer. Same error.

The requirments.txt I'm using to create my packages folder or my Layer contains:

My Layer is created using a zip file which has a path \python\lib\python3.9\site-packages\

(I created the layer by running the below command: pip install . -t \python\lib\python3.9\site-packages)

Wierdly when I run this locally, it works fine inside my virtual environment.

Im developing in Windows.

Solution

  • I might have missed trying this but many might not know that this could be an issue. So I'll answer the question myself.

    Option 1:

    Usually, if we want to deploy external libraries (libraries which AWS does not provide by default in the Lambda instances), we use Lambda deployment packages.

    A lambda deployment package is a zip file that contains the Lambda code at its root and the python libraries that the code needs in a folder named packages.

    mypackage.zip
|
|--> packages
|    |
|    |-->pip
|    |-->smbprotocol
|    |-->..
|--> myscript.py

    Option 2:

    Or some people also want to use a Lambda Layer which they create using their dependencies and then add that layer to the lambda function. (Mostly if the same libraries are used by multiple lambda functions)

    Below is the required structure for the zip needed for the layer.

    mylayer.zip
|
|--> python
     |
     |-->lib
         |
         |-->python3.9 <your version of python>
             |
             |-->site-packages
                 |
                 |--> <pip should download all required libraries here>

    And usually, python libraries are platform-independent. But in a few cases, it isn't. And in such cases, the pip install needs to be done in an AmazonLinux container before adding then to the zip.

    And if a library is not platform-independent then in both Options, you must run the pip install in an AmazonLinux container and create a package using those libraries. Why AmazonLinux? Because Lambda instances run in these containers.

    Hope this helps someone someday. :)