amazon-web-servicesaws-lambdapython-3.7serverless-frameworkdlib

Package dlib in AWS Lambda using Serverless and python 3.7


I am trying to deploy an AWS Lambda function using Serverless framework with python3.7 runtime and I need to package dlib as a dependency and import inside the lambda function. Any ideas what's the easiest way to make dlib work on python3.7 and package successfully using the Serverless framework? TIA...

UPDATE: I already added dlib==19.9.0 to my requirements.txt, I am using the Serverless plugin serverless-python-requirements and I deploy using bitbucket pipeline based on an ubuntu image with python & node base. I also install cmake in the pipeline script since dlib needs it to compile as I've seen.

The pipeline fails with:

Container 'Build' exceeded memory limit.

and the dlib compiling stops at 77% with the following recurring deprecation warnings until the pipeline exceeds memory and stops with failure:

 Scanning dependencies of target dlib
.
.

 [ 77%] Building CXX object CMakeFiles/dlib_python.dir/src/other.cpp.o
    In file included from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/cast.h:16,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/attr.h:13,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/pybind11.h:43,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/../dlib/python/pybind_utils.h:6,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/../dlib/python.h:6,
                     from /tmp/pip-install-nx1hok9_/dlib/tools/python/src/basic.cpp:3:
    /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/detail/internals.h:82:34: warning: ‘int PyThread_create_key()’ is deprecated [-Wdeprecated-declarations]
         decltype(PyThread_create_key()) tstate = 0; // Usually an int but a long on Cygwin64 with Python 3.x
                                      ^
    In file included from /usr/local/include/python3.7m/pystate.h:11,
                     from /usr/local/include/python3.7m/traceback.h:8,
                     from /usr/local/include/python3.7m/Python.h:119,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/detail/common.h:111,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/pytypes.h:12,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/cast.h:13,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/attr.h:13,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/pybind11.h:43,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/../dlib/python/pybind_utils.h:6,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/../dlib/python.h:6,
                     from /tmp/pip-install-nx1hok9_/dlib/tools/python/src/basic.cpp:3:
    /usr/local/include/python3.7m/pythread.h:95:17: note: declared here
     PyAPI_FUNC(int) PyThread_create_key(void) Py_DEPRECATED(3.7);
                     ^~~~~~~~~~~~~~~~~~~
    In file included from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/cast.h:16,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/attr.h:13,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/pybind11.h:43,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/../dlib/python/pybind_utils.h:6,
                     from /tmp/pip-install-nx1hok9_/dlib/dlib/../dlib/python.h:6,
                     from /tmp/pip-install-nx1hok9_/dlib/tools/python/src/basic.cpp:3:
    /tmp/pip-install-nx1hok9_/dlib/dlib/external/pybind11/include/pybind11/detail/internals.h:82:34: warning: ‘int PyThread_create_key()’ is deprecated [-Wdeprecated-declarations]
         decltype(PyThread_create_key()) tstate = 0; // Usually an int but a long on Cygwin64 with Python 3.x
                                      ^
    In file included from /usr/local/include/python3.7m/pystate.h:11,


Solution

  • ok I solved this by increasing the pipeline size to 2x and it succeeded