I want to limit the number of times that AWS Parameter Store is called in my AWS Lambda. Using a global variable, I'm caching a Parameter Store value on the first call to Parameter Store.
main.py
import os
import boto3
redis_password = None
def get_redis_password():
global redis_password
if not redis_password:
client = boto3.client("ssm")
redis_password = client.get_parameter(
Name=f"{os.environ["ENV"]}.redis-cache.password",
WithDecryption=True
)
return redis_password["Parameter"]["Value"]
def lambda_handler(event, context):
get_redis_password()
However, if I want to cache multiple Parameter Store values, I must create multiple global variables and if not [INSERT_GLOBAL_VARIABLE]
checks. For example:
main.py
import os
import boto3
redis_password = None
another_parameter_store_value = None
def get_redis_password():
global redis_password
if not redis_password:
client = boto3.client("ssm")
redis_password = client.get_parameter(
Name=f"{os.environ["ENV"]}.redis-cache.password",
WithDecryption=True
)
return redis_password["Parameter"]["Value"]
def get_another_parameter_store_value():
global another_parameter_store_value
if not another_parameter_store_value:
client = boto3.client("ssm")
another_parameter_store_value = client.get_parameter(
Name=f"{os.environ["ENV"]}.another.parameter.store.key",
WithDecryption=True
)
return redis_password["Parameter"]["Value"]
def lambda_handler(event, context):
get_redis_password()
get_another_parameter_store_value()
In an attempt to solve this issue, I've created a Parameter Store utility.
parameter_util.py
import os
import boto3
class ParameterUtil:
def __init__(self):
self.boto_client = boto3.client("ssm")
def get_parameter(self, parameter_path):
response = self.boto_client.get_parameter(
Name=f"{os.environ['ENV']}.{parameter_path}", WithDecryption=True
)
return response["Parameter"]["Value"]
My theory is that by instantiating the AWS Boto client as an instance variable, it will cache the entire Boto client object. Then get_parameter
will be called using the cached Boto client. For example:
main.py
import os
import boto3
from parameter_util import ParameterUtil
redis_password = None
def get_redis_password():
global redis_password
if not redis_password:
client = boto3.client("ssm")
redis_password = client.get_parameter(
Name=f"{os.environ["ENV"]}.redis-cache.password",
WithDecryption=True
)
return redis_password["Parameter"]["Value"]
def lambda_handler(event, context):
param_util = ParameterUtil()
param_util.get_parameter(".redis-cache.password")
param_util.get_parameter(".another.parameter.store.key")
However, I'm not really sure if this solves the issue.
Does caching the Boto client result in only one call per parameter to the Parameter Store when get_parameter
is called? Or am I optimizing in the wrong place?
Your original code won't work because param_util
is a local variable that will go out of scope for every Lambda call.
You can use the built-in @functools.lru_cache
to create a simple function that handles any parameter. It will cache the return values for you based on the input of the function (Python 3.2+).
Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.
Example:
ssm_client = boto3.client("ssm")
@lru_cache(maxsize=None)
def get_param(name):
return ssm_client.get_parameter(
Name=f"{os.environ['ENV']}.{name}",
WithDecryption=True
)["Parameter"]["Value"]
def lambda_handler(event, context):
redis_password = get_param("redis-cache.password")
another_parameter_store_key = get_param("another.parameter.store.key")