I'm fairly new to scala / java / sbt and this is my first time debugging a discrepancy between code ran in 'scala' through sbt and the same code ran from a compiled jar. I have some Scala code which utilizes the AWS Java SDK to make some S3 requests. When I run my code in "Scala" using sbt or Intellij the code succeeds as intended. If I build a fat jar using sbt and run the same code in "Java" using the jar, the code fails. I debugged the failure and can attribute it to missing ExecutionInterceptors in the interceptor chain:
// Logging the code ran from 'scala'
2022-12-21 14:09:33,861 [main] DEBUG software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain - Creating an interceptor chain that will apply interceptors in the following order:
[software.amazon.awssdk.core.internal.interceptor.HttpChecksumRequiredInterceptor@5965844d,
software.amazon.awssdk.core.internal.interceptor.SyncHttpChecksumInTrailerInterceptor@6d4a65c6,
software.amazon.awssdk.core.internal.interceptor.HttpChecksumValidationInterceptor@aa004a0,
software.amazon.awssdk.core.internal.interceptor.AsyncRequestBodyHttpChecksumTrailerInterceptor@4c98a6d5,
software.amazon.awssdk.core.internal.interceptor.HttpChecksumInHeaderInterceptor@392a04e7,
software.amazon.awssdk.awscore.interceptor.HelpfulUnknownHostExceptionInterceptor@7f02251,
software.amazon.awssdk.awscore.eventstream.EventStreamInitialRequestInterceptor@dffa30b,
software.amazon.awssdk.awscore.interceptor.TraceIdExecutionInterceptor@4d8126f,
software.amazon.awssdk.services.s3.internal.handlers.EnableChunkedEncodingInterceptor@6d3c232f,
software.amazon.awssdk.services.s3.internal.handlers.DisableDoubleUrlEncodingInterceptor@6b587673,
software.amazon.awssdk.services.s3.internal.handlers.EnableTrailingChecksumInterceptor@1bcf67e8,
software.amazon.awssdk.services.s3.internal.handlers.CreateMultipartUploadRequestInterceptor@5f404594,
software.amazon.awssdk.services.s3.internal.handlers.GetObjectInterceptor@53692008,
software.amazon.awssdk.services.s3.internal.handlers.ExceptionTranslationInterceptor@7b2a3ff8,
software.amazon.awssdk.transfer.s3.internal.ApplyUserAgentInterceptor@1bbae752,
software.amazon.awssdk.services.s3.internal.handlers.AsyncChecksumValidationInterceptor@460b6d54,
software.amazon.awssdk.services.s3.internal.handlers.EndpointAddressInterceptor@5cf87cfd,
software.amazon.awssdk.services.s3.internal.handlers.SignerOverrideInterceptor@76075d65,
software.amazon.awssdk.services.s3.internal.handlers.GetBucketPolicyInterceptor@3a4ba480,
software.amazon.awssdk.services.s3.internal.handlers.PutObjectInterceptor@27b71f50,
software.amazon.awssdk.services.s3.internal.handlers.SyncChecksumValidationInterceptor@383790cf,
software.amazon.awssdk.services.s3.internal.handlers.DecodeUrlEncodedResponseInterceptor@74971ed9,
software.amazon.awssdk.services.s3.internal.handlers.CreateBucketInterceptor@131fcb6f,
software.amazon.awssdk.services.s3.internal.handlers.CopySourceInterceptor@ccd1bc3]
// Logging the same code ran from 'java'
2022-12-21 14:15:06,281 [main] DEBUG software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain - Creating an interceptor chain that will apply interceptors in the following order:
[software.amazon.awssdk.core.internal.interceptor.HttpChecksumRequiredInterceptor@147e2ae7,
software.amazon.awssdk.core.internal.interceptor.SyncHttpChecksumInTrailerInterceptor@448c8166,
oftware.amazon.awssdk.core.internal.interceptor.HttpChecksumValidationInterceptor@4470fbd6,
software.amazon.awssdk.core.internal.interceptor.AsyncRequestBodyHttpChecksumTrailerInterceptor@15d49048,
software.amazon.awssdk.core.internal.interceptor.HttpChecksumInHeaderInterceptor@7098b907,
software.amazon.awssdk.awscore.interceptor.HelpfulUnknownHostExceptionInterceptor@503f91c3,
software.amazon.awssdk.awscore.eventstream.EventStreamInitialRequestInterceptor@13526e59,
software.amazon.awssdk.awscore.interceptor.TraceIdExecutionInterceptor@2235eaab,
software.amazon.awssdk.transfer.s3.internal.ApplyUserAgentInterceptor@17503f6b]
All of the interceptors of the type software.amazon.awssdk.services.s3.internal.handlers
are missing from the code ran in the jar.
I'm not sure if this is directly related. My build.sbt
has the following merge strategies:
assembly / assemblyMergeStrategy := {
// Don't discard Geotools META-INF service files
case PathList("META-INF", xs @ _*) =>
(xs map { _.toLowerCase }) match {
case ("manifest.mf" :: Nil) | ("index.list" :: Nil) | ("dependencies" :: Nil) =>
MergeStrategy.discard
case ps @ (x :: xs) if ps.last.endsWith(".sf") || ps.last.endsWith(".dsa") =>
MergeStrategy.discard
case "plexus" :: xs =>
MergeStrategy.discard
case "services" :: xs =>
MergeStrategy.filterDistinctLines
case ("spring.schemas" :: Nil) | ("spring.handlers" :: Nil) =>
MergeStrategy.filterDistinctLines
case _ => MergeStrategy.discard
}
case PathList("reference.conf", xs @ _*) => MergeStrategy.concat
case PathList("version.conf", xs @ _*) => MergeStrategy.concat
case x => MergeStrategy.first
},
When debugging compiling my jar with sbt assembly
I noticed this line:
[debug] Merging 'software/amazon/awssdk/services/s3/execution.interceptors' with strategy 'first'
Is this related to why i'm missing the interceptors? or might there be some other cause for missing these handlers, such as a missing dependency?
I solved this question, the issue was that the aws s3 interceptors were getting merged out, I added this line to the merge strategy and it solved the problem:
case PathList(
"software",
"amazon",
"awssdk",
"services",
"s3",
"execution.interceptors",
xs @ _*
) =>
MergeStrategy.concat