[SOLVED] Defining AWS/boto3 clients and resources outside of functions with unit tests

Defining AWS/boto3 clients and resources outside of functions with unit tests

I am writing some unit tests using pytest for some functions that are called using AWS Lambda. I am using boto3 to interact with AWS, and I'm mocking using moto. Where I am using a resource or client from boto3, I've defined them outside of functions as I understand that allows for them to be reused across warm starts.

However, when I try to run unit tests with moto mocking, I encounter an authentication error as my code tries to connect to AWS proper and ignores my mocked creds. This error disappears if I move the resource or client definition into the functions themselves, but then I'd be creating a new one every time. Is there a neat way to get around this issue without having to create a new resource/client object in every function that uses the service? Perhaps by defining a singleton class for each one or something?

The function code looks something like this at the moment, and this throws the error:

import boto3

SM_CLIENT = boto3.client(service_name="secretsmanager")

def get_secret_key_string():
    return SM_CLIENT.get_secret_value(SecretId="my_key")["SecretString"]

This however does not throw the error:

import boto3

def get_secret_key_string():
    SM_CLIENT = boto3.client(service_name="secretsmanager")
    return SM_CLIENT.get_secret_value(SecretId="my_key")["SecretString"]

My conftest.py file has this in it:

import os
import pytest
from moto import mock_secretsmanager

@pytest.fixture(scope="session", autouse=True)
def aws_credentials():
    """Mocked AWS Credentials for moto"""
    os.environ["AWS_ACCESS_KEY_ID"] = "testing"
    os.environ["AWS_SECRET_ACCESS_KEY"] = "testing"
    os.environ["AWS_SECURITY_TOKEN"] = "testing"
    os.environ["AWS_SESSION_TOKEN"] = "testing"
    os.environ["AWS_DEFAULT_REGION"] = "us-east-1"

@pytest.fixture(scope="session")
def secrets_client(aws_credentials):
    with mock_secretsmanager():
        client = boto3.client("secretsmanager")
        client.create_secret(Name="my_key", SecretString="abcde")
        yield client

And my test code looks like this:

from my_module import get_secret_key_string

def test_get_secret_key_string(secrets_client):
    secret_string = get_secret_key_string()
    assert secret_string == "abcde"

Solution

You can explicitly patch clients that are created outside the mock_aws-scope.

from moto.core import patch_client, patch_resource
patch_client(SM_CLIENT)

See this section of the documentation:

http://docs.getmoto.org/en/latest/docs/getting_started.html#patching-the-client-or-resource