I am trying to create a simple MediaConnect job with Python.
My pipeline is simple. S3Put
triggers a Python lambda
, and I am trying to create a simple job.
I created a simple job using AWS Console and the json job is this -
{
"Queue": "arn:aws:mediaconvert:ap-south-1:----:queues/Default",
"UserMetadata": {},
"Role": "arn:aws:iam::----:role/mediaConverterRole",
"Settings": {
"TimecodeConfig": {
"Source": "ZEROBASED"
},
"OutputGroups": [
{
"Name": "File Group",
"Outputs": [
{
"Preset": "System-Generic_Hd_Mp4_Av1_Aac_16x9_640x360p_24Hz_250Kbps_Qvbr_Vq6",
"Extension": ".mp4",
"NameModifier": "converted"
}
],
"OutputGroupSettings": {
"Type": "FILE_GROUP_SETTINGS",
"FileGroupSettings": {
"Destination": "s3://----/"
}
}
}
],
"Inputs": [
{
"AudioSelectors": {
"Audio Selector 1": {
"DefaultSelection": "DEFAULT"
}
},
"VideoSelector": {},
"TimecodeSource": "ZEROBASED",
"FileInput": "s3://----/videos/sample786.mp4"
}
]
},
"AccelerationSettings": {
"Mode": "DISABLED"
},
"StatusUpdateInterval": "SECONDS_60",
"Priority": 0
}
Please note that the Role
worked fine while using on AWS console. So far this is ok.
Now coming to my pipeline with s3Put -> Python Lambda -> MediaConnect
, the infrastructure is written using Terraform
. My iam.tf
file -
# create a role
# reseource_type - resource_name
resource "aws_iam_role" "lambda_role" {
name = "${local.resource_component}-lambda-role"
assume_role_policy = jsonencode({
"Version": "2012-10-17",
"Statement": [{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
},
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "mediaconvert.amazonaws.com"
},
"Sid": "",
"Effect": "Allow",
}
]
})
}
# create policy
resource "aws_iam_policy" "policy" {
name = "${local.resource_component}-lambda-policy"
policy = jsonencode({
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": "arn:aws:s3:::*"
}
]
})
}
# attach policy to the role
resource "aws_iam_role_policy_attachment" "policy_attachment" {
role = "${aws_iam_role.lambda_role.name}"
policy_arn = "${aws_iam_policy.policy.arn}"
}
The lambda code gets triggered by S3Put
successfully. But the lambda throws error -
(AccessDeniedException) when calling the CreateJob operation: User: arn:aws:sts::---:assumed-role/vidstream-inputVideoProcessor-lambda-role/vidstream-inputVideoProcessor is not authorized to perform: iam:PassRole on resource: arn:aws:iam::---:role/mediaConverterRole
I have tried to find boto3
simple examples but nothing simpler is found online.
The lambda Python Code is here -
import json
import logging
import boto3
# initialize logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)
def handler(event, context):
# get input bucket
input_bucket_name = event['Records'][0]['s3']['bucket']['name']
# get file/object name
media_object = event['Records'][0]['s3']['object']['key']
# open json mediaconvert template
with open("job.json", "r") as jsonfile:
job_object = json.load(jsonfile)
# prepare data for mediaconvert job
input_file = f's3://{input_bucket_name}/{media_object}'
# edit job object
job_object['Settings']['Inputs'][0]['FileInput'] = input_file
# updated job object
logger.info("updated job object")
# Create MediaConvert client
mediaconvert_client = boto3.client('mediaconvert')
try:
# try to create a job
mediaconvert_client.create_job(**job_object)
except Exception as e:
logger.error(e)
return {
'statusCode': 200,
'body': json.dumps(event)
}
The boto3 MediaConvert
documentation is provided by AWS
I am at a loss, no idea what to do. Is there any simpler example anyone can help me with? I just need to create a simple job with Lambda that works, no complexity.
Any kind of help will be highly appreciated.
Okay I solved this issue by putting iam:PassRole
to lambda policy.
{
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": "*"
}
So the updated iam.tf
file is -
# create a role
# reseource_type - resource_name
resource "aws_iam_role" "lambda_role" {
name = "${local.resource_component}-lambda-role"
assume_role_policy = jsonencode({
"Version": "2012-10-17",
"Statement": [{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
},
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "mediaconvert.amazonaws.com"
},
"Sid": "",
"Effect": "Allow",
}
]
})
}
# create policy
resource "aws_iam_policy" "policy" {
name = "${local.resource_component}-lambda-policy"
policy = jsonencode({
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": "*"
}
]
})
}
# attach policy to the role
resource "aws_iam_role_policy_attachment" "policy_attachment" {
role = "${aws_iam_role.lambda_role.name}"
policy_arn = "${aws_iam_policy.policy.arn}"
}
I first added this to lambda policy from aws console. After that worked I added this on my tf file. Be careful when editing something on console while the main infrastructure is written in IACs such as Terraform
, this might cause drift if you forget what you have done.