pythonaws-lambdapostgisogrosgeo

Python code in AWS Lambda to load Shapefile to PostGIS(RDS)

I am new to GIS implementation. I am trying to develop AWS Lambda Code in Python to Load Shape File Dynamically.

I developed the code after doing some research and it is perfectly working on my local.

But the same code is troubling when I am trying to run in AWS Lambda.

I have added libraries(Lambda Layers) for 'OSGEO/GDAL' in AWS Lambda and tested it by calling import and it's working fine.

Following is the code :

import os
import subprocess
import boto3
import urllib.parse
from osgeo import gdal
from osgeo import ogr

s3 = boto3.client('s3')

def lambda_handler(event, context):  

    bucket = event['Records'][0]['s3']['bucket']['name']
    s3key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'], encoding='utf-8')

    # input shapefile
    input_shp = ('s3://' + bucket + '/' + s3key)


    # database options
    db_schema = "SCHEMA=public"
    overwrite_option = "OVERWRITE=YES"
    geom_type = "MULTILINESTRING"
    output_format = "PostgreSQL"
    
    # database connection string
    db_connection = """PG:host=<RDS host name> port=5432 user=<RDS User Name> dbname= postgres password=<RDS Password>"""
    
    
    # call ogr2ogr from python
    subprocess.call(["ogr2ogr", "-lco", db_schema, "-lco", overwrite_option, "-nlt", geom_type, "-f", output_format, db_connection, input_shp])

Error Message is :

[ERROR] FileNotFoundError: [Errno 2] No such file or directory: 'ogr2ogr': 'ogr2ogr'

The same code is working fine on my local with only difference that instead of S3 I am providing hard-coded path where shape file is stored on my local machine.

Any suggestions!

Solution

  • GDAL/Osgeo is a bit problematic when it comes to Python.

    But this works on me. You can use this :https://github.com/developmentseed/geolambda

    You just need to add these layers to your lambda function.