javaaws-lambdabuild.gradlegraalvmmicronaut-aws

Micronaut GraalVM Sendgrid AWS lambda application keeps getting a 400 returned from Sendgrid when trying to send email


I built a simple emailing application using micronaut, java, and sendgrid. Using it as a basic java application deployed to AWS works, emails send fine. I have created another lambda to try using the GraalVM capabilities. I simply followed the Micronaut docs guid, everything compiles, and builds using ./gradlew buildNativeLambda. Running the test option in the AWS console works and the application returns no errors, however sendgrid is sending a 400 back. I feel I am missing something basic. I have tried adding everything I can think of to the resources/META-INF/native-image/reflect-config.json . Tried the different ways shown in the micronaut docs to run the lambda , i.e. using a Controller method, the FunctionRequestHandler , and FunctionLambdaRuntime. With each the lambda will work and it returns a string or APIGatewayProxyResponseEvent no problem. There seems to be something with building the email and I feel I have been just throwing sh*t at the wall at this point and nothing is sticking, no matter what i change I am not getting new error codes or anything to push me in the right direction.

    plugins {
        id("com.github.johnrengelman.shadow") version "8.1.1"
        id("io.micronaut.application") version "4.4.2"
        id("com.diffplug.spotless") version "6.23.3"
        id("io.micronaut.aot") version "4.4.2"
    }
    
    version = "0.1"
    group = "example.micronaut"
    
    repositories {
        mavenCentral()
    }
    
    dependencies {
        annotationProcessor("io.micronaut:micronaut-http-validation")
        annotationProcessor("io.micronaut.serde:micronaut-serde-processor")
        implementation("io.micronaut.email:micronaut-email-sendgrid")
        implementation("io.micronaut:micronaut-http-client-jdk")
        implementation("jakarta.mail:jakarta.mail-api:2.1.3")
        implementation("io.micronaut.aws:micronaut-aws-lambda-events-serde")
        implementation("io.micronaut.serde:micronaut-serde-jackson")
        runtimeOnly("org.yaml:snakeyaml")
        runtimeOnly("ch.qos.logback:logback-classic")
    }
    
    
    application {
        mainClass = "example.micronaut.Application"
    }
    java {
        sourceCompatibility = JavaVersion.toVersion("17")
        targetCompatibility = JavaVersion.toVersion("17")
    }
    
    shadowJar {
        // Ensure resources are included
        mergeServiceFiles()
        include 'EmailTemplate/**'
    }
    
    sourceSets {
        main {
            resources {
                srcDirs = ['src/main/resources']
    //            include([ '**/*.properties', '**/*.yml', '**/*.json', '**/*.png', '**/*.html', '**/*.css','**/*.JPG'])
            }
        }
    }
    
    graalvmNative {
        toolchainDetection = false
    
        binaries {
            main {
                javaLauncher = javaToolchains.launcherFor {
                    languageVersion = JavaLanguageVersion.of(17)
                    vendor = JvmVendorSpec.matching("GraalVM Community")
                }
                resources.autodetect()
                metadataRepository { enabled = true }
                imageName.set('graal-mail')
                buildArgs.add('--verbose')
                buildArgs.add('--initialize-at-build-time=kotlin.coroutines.intrinsics.CoroutineSingletons')
                buildArgs.add('--initialize-at-run-time=reactor.core.publisher.Traces$StackWalkerCallSiteSupplierFactory')
                buildArgs.add('--initialize-at-run-time=reactor.core.publisher.Traces$ExceptionCallSiteSupplierFactory')
            }
        }
    }
    
    micronaut {
        runtime("lambda_provided")
        testRuntime("junit5")
        processing {
            incremental(true)
            annotations("example.micronaut.*")
        }
        aot {
            // Please review carefully the optimizations enabled below
            // Check https://micronaut-projects.github.io/micronaut-aot/latest/guide/ for more details
            optimizeServiceLoading = false
            convertYamlToJava = false
            precomputeOperations = true
            cacheEnvironment = true
            optimizeClassLoading = true
            deduceEnvironment = true
            optimizeNetty = true
            replaceLogbackXml = true
        }
    }
    
    
    tasks.named("dockerfileNative") {
        baseImage = "amazonlinux:2023"
        jdkVersion = "17"
        args(
                "-XX:MaximumHeapSizePercent=80",
                "-Dio.netty.allocator.numDirectArenas=0",
                "-Dio.netty.noPreferDirect=true"
        )
    }
    
    spotless {
        java {
            licenseHeaderFile(file("LICENSEHEADER"))
        }
    } 
    package example.micronaut.services
    
    import com.sendgrid.Response;
    import example.micronaut.Util.MimeType;
    import example.micronaut.Util.UtilMailService;
    import io.micronaut.context.annotation.Value;
    import io.micronaut.core.annotation.ReflectiveAccess;
    import io.micronaut.email.BodyType;
    import io.micronaut.email.Email;
    import io.micronaut.email.sendgrid.SendgridEmailSender;
    import jakarta.inject.Inject;
    import jakarta.inject.Singleton;
    import jakarta.mail.internet.MimeBodyPart;
    
    import java.util.Optional;
    import java.util.concurrent.atomic.AtomicInteger;
    
    @Singleton
    @ReflectiveAccess
    public class TestCampaign {
        private final UtilMailService utilMailService;
        private final SendgridEmailSender sendgridEmailSender;
    
        @Value("${micronaut.email.from.email}")
        private String fromEmail;
    
        @Inject
        public TestCampaign(SendgridEmailSender sendgridEmailSender, UtilMailService utilMailService) {
            this.sendgridEmailSender = sendgridEmailSender;
            this.utilMailService = utilMailService;
        }
    
        public Response sendTestEmail() throws Exception {
            AtomicInteger index = new AtomicInteger(0);
            Email.Builder emailBuilder = getEmailBuilder();
    
            utilMailService.getContacts("EmailTemplate/EmailListTest.json").forEach(contact -> {
                if (index.getAndIncrement() == 0) {
                    emailBuilder.to(contact);
                } else {
                    emailBuilder.bcc(contact);
                }
            });
    
            return sendgridEmailSender.send(emailBuilder.build());
        }
    
        private Email.Builder getEmailBuilder() throws Exception {
            Optional<String> bodyOption = utilMailService.readHtmlFile("EmailTemplate/StdEmail.html");
            String body = bodyOption.orElse("Be Aware of Your Prescriptions at work");
            return Email.builder()
                    .from(fromEmail)
                    .subject("subject")
                    .body(body, BodyType.HTML)
                    .attachment(utilMailService.buildAttachment("EmailTemplate/Meds1.png", "meds1.png", MimeBodyPart.ATTACHMENT, MimeType.IMAGE_PNG).build())
                    .attachment(utilMailService.buildAttachment("EmailTemplate/Meds2.png", "meds2.png", MimeBodyPart.ATTACHMENT, MimeType.IMAGE_PNG).build())
                    .attachment(utilMailService.buildAttachment("EmailTemplate/Meds3.JPG", "meds3.JPG", MimeBodyPart.ATTACHMENT, MimeType.IMAGE_JPEG).build())
                    .attachment(utilMailService.buildAttachment("EmailTemplate/AllMeds.png", "AllMeds.png", MimeBodyPart
    package example.micronaut.Util;
    
    import io.micronaut.core.annotation.NonNull;
    import io.micronaut.core.annotation.Nullable;
    import io.micronaut.core.annotation.ReflectiveAccess;
    import io.micronaut.core.io.IOUtils;
    import io.micronaut.core.io.ResourceResolver;
    import io.micronaut.core.type.Argument;
    import io.micronaut.email.Attachment;
    import io.micronaut.email.Contact;
    import io.micronaut.serde.ObjectMapper;
    import jakarta.inject.Inject;
    import jakarta.inject.Singleton;
    import jakarta.mail.internet.MimeBodyPart;
    
    import java.io.BufferedReader;
    import java.io.IOException;
    import java.io.InputStream;
    import java.io.InputStreamReader;
    import java.net.URL;
    import java.util.*;
    
    @Singleton
    @ReflectiveAccess
    public class UtilMailService {
        private final ResourceResolver resourceResolver;
        private final ObjectMapper objectMapper;
    
    
        @Inject
        public UtilMailService(ResourceResolver resourceResolver, ObjectMapper objectMapper) {
            this.objectMapper = objectMapper;
            this.resourceResolver = resourceResolver;
        }
    
        public @NonNull Attachment.Builder buildAttachment(String path, String name, String disposition, MimeType type) throws Exception {
            Optional<byte[]> fileBytes = getClasspathResourceAsBytes(path);
    
            if (fileBytes.isEmpty()) {
                throw new IllegalArgumentException("File not found! " + path);
            }
    
            Attachment.Builder newAttachment = Attachment.builder().filename(name).contentType(type.getMimeType()).content(fileBytes.get());
    
            if (Objects.equals(disposition, MimeBodyPart.INLINE)) {
    
                newAttachment.id(name).disposition(disposition);
            }
    
            return newAttachment;
        }
    
        public Optional<byte[]> getClasspathResourceAsBytes(String path) throws Exception {
            Optional<URL> url = resourceResolver.getResource("classpath:" + path);
            if (url.isPresent()) {
                try (InputStream inputStream = url.get().openStream()) {
                    return Optional.of(inputStream.readAllBytes());
                }
            }
            else {
                return Optional.empty();
            }
        }
    
        public List<Contact> getContacts(String path) throws IOException {
            List<Contact> contactList = new ArrayList<>();
            Map<String, String> contactMap = readJsonFileToMap(path).orElse(Map.of("sender@gmail.com", "crash"));
            contactMap.forEach((key, value) -> {
                contactList.add(new Contact(key, value));
            });
    
            return contactList;
        }
    
        public @Nullable Optional<Map<String, String>> readJsonFileToMap(String resourcePath) throws IOException {
            Optional<URL> url = resourceResolver.getResource("classpath:" + resourcePath);
            if (url.isPresent()) {
                try (InputStream inputStream = url.get().openStream()) {
                    return Optional.of(objectMapper.readValue(inputStream.readAllBytes(), Argument.mapOf(String.class, String.class)));
                }
            }
            else {
                return Optional.empty();
            }
        }
    
        public Optional<String> readHtmlFile(String path) throws Exception {
            Optional<URL> url = resourceResolver.getResource("classpath:" + path);
            if (url.isPresent()) {
                return Optional.of(IOUtils.readText(new BufferedReader(new InputStreamReader(url.get().openStream()))));
            }
            else {
                return Optional.empty();
            }
        }
    
        public Optional<String> getClasspathResourceAsText(String path) throws Exception {
            Optional<URL> url = resourceResolver.getResource("classpath:" + path);
            if (url.isPresent()) {
                return Optional.of(IOUtils.readText(new BufferedReader(new InputStreamReader(url.get().openStream()))));
            }
            else {
                return Optional.empty();
            }
        }
    }
package example.micronaut;

import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent;
import com.sendgrid.Response;
import example.micronaut.services.EmailSendingService;
import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Get;
import io.micronaut.http.annotation.QueryValue;
import io.micronaut.serde.ObjectMapper;
import jakarta.inject.Inject;

import java.util.Collections;

@Controller
public class HomeController {
    private final EmailSendingService emailSendingService;
    @Inject
    ObjectMapper objectMapper;
    APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent();

    @Inject
    public HomeController(EmailSendingService emailSendingService) {
        this.emailSendingService = emailSendingService;
    }

    @Get
    public APIGatewayProxyResponseEvent index(@QueryValue(defaultValue = "test") String campaign) {
        try {

            Response sendGridResponse = emailSendingService.sendCustomizedEmail(campaign);
            String json = new String(objectMapper.writeValueAsBytes(Collections.singletonMap("message", response.getHeaders())));
            response.setStatusCode(sendGridResponse.getStatusCode());
            response.setBody(json);
        }
        catch (Exception e) {
            response.setStatusCode(500);
            response.setBody(String.valueOf(e.getMessage()));
        }

        return response;
    }
}

enter image description here

-- execution log

    {
      "statusCode": 200,
      "headers": {
        "Date": "Sat, 14 Sep 2024 18:05:23 GMT",
        "Content-Type": "application/json"
      },
      "multiValueHeaders": {
        "Date": [
          "Sat, 14 Sep 2024 18:05:23 GMT"
        ],
        "Content-Type": [
          "application/json"
        ]
      },
      "body": "{\"statusCode\":400,\"body\":\"{\\\"message\\\":null}\"}",
      "isBase64Encoded": false
    }

-- log output

    START RequestId: b984c570-f7af-4d4c-a929-0ae8cf1fdcd7 Version: $LATEST
     [36m18:05:23.612 [0;39m  [1;30m[main] [0;39m  [34mINFO  [0;39m  [35mi.m.e.sendgrid.SendgridEmailSender [0;39m - Status Code: 400
     [36m18:05:23.612 [0;39m  [1;30m[main] [0;39m  [34mINFO  [0;39m  [35mi.m.e.sendgrid.SendgridEmailSender [0;39m - Body: {"errors":[{"message":"The from object must be provided for every email send. It is an object that requires the email parameter, but may also contain a name parameter.  e.g. {\"email\" : \"example@example.com\"}  or {\"email\" : \"example@example.com\", \"name\" : \"Example Recipient\"}.","field":"from.email","help":"http://sendgrid.com/docs/API_Reference/Web_API_v3/Mail/errors.html#message.from"},{"message":"The personalizations field is required and must have at least one personalization.","field":"personalizations","help":"http://sendgrid.com/docs/API_Reference/Web_API_v3/Mail/errors.html#-Personalizations-Errors"},{"message":"Unless a valid template_id is provided, the content parameter is required. There must be at least one defined content block. We typically suggest both text/plain and text/html blocks are included, but only one block is required.","field":"content","help":"http://sendgrid.com/docs/API_Reference/Web_API_v3/Mail/errors.html#message.content"}]}
     [36m18:05:23.612 [0;39m  [1;30m[main] [0;39m  [34mINFO  [0;39m  [35mi.m.e.sendgrid.SendgridEmailSender [0;39m - Headers {Strict-Transport-Security=max-age=600; includeSubDomains, Server=nginx, Access-Control-Allow-Origin=https://sendgrid.api-docs.io, Access-Control-Allow-Methods=POST, Connection=keep-alive, X-No-CORS-Reason=https://sendgrid.com/docs/Classroom/Basics/API/cors.html, Content-Length=980, Access-Control-Max-Age=600, Date=Sat, 14 Sep 2024 18:05:23 GMT, Access-Control-Allow-Headers=Authorization, Content-Type, On-behalf-of, x-sg-elas-acl, Content-Type=application/json}
    END RequestId: b984c570-f7af-4d4c-a929-0ae8cf1fdcd7
    REPORT RequestId: b984c570-f7af-4d4c-a929-0ae8cf1fdcd7  Duration: 2722.27 ms    Billed Duration: 2723 ms    Memory Size: 128 MB Max Memory Used: 113 MB 

Like I said the application works when not deployed as GraalVM. Any help or something else to try is greatly appreciated

I have tried adding everything I can think of to the resources/META-INF/native-image/reflect-config.json . Tried the different ways shown in the micronaut docs to run the lambda , i.e. using a Controller method, the FunctionRequestHandler , and FunctionLambdaRuntime. With each the lambda will work and it returns a string or APIGatewayProxyResponseEvent no problem. There seems to be something with building the email


Solution

  • I was thinking that the io.micronaut.email.Email class that is used to build the email was using reflection somewhere and it was not registering once compiled to GraalVM. After walking through the code of how io.micronaut.email.sendgrid.SendgridEmailSender was working, the reflection issue was coming from io.micronaut.email.sendgrid.SendgridEmailComposer, particularly in the methods for creating the sendgrid Objects Personalization and Content.

    User the GraalConfig.java file for reflection configuration. Explained here on the micronaut page Reflection Metadata

    @ReflectionConfig(type = Content.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Content.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Content.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Personalization.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Personalization.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Personalization.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Mail.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Mail.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Mail.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Attachments.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Attachments.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Attachments.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Email.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Email.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    @ReflectionConfig(type = Email.class, accessType = TypeHint.AccessType.ALL_DECLARED_METHODS)
    class GraalConfig {
    }