[SOLVED] Boto3 - How to combine TransferConfig and Config?

Boto3 - How to combine TransferConfig and Config?

I have this Python code and I am trying to find a way to combine two configs:

...
from boto3.s3.transfer import TransferConfig
from botocore.client import Config
...

transfer_config = TransferConfig(max_concurrency=XXX, ...)
config = Config(retries=XX,region_name=XX, ...)

s3 = boto3.Session().resource("s3", config = [__HOW I can combibe "transfer_config" and "config" here?])

I need two configurations because, for example, max_concurrency cant be applied to Configandsignature_versiontoTransferConfig`. I need all these paremeters (and more)

Solution

They do not get combined as they are used for different reasons. Where "botocore.client.Config" is for configuring the client itself regardless of the resource and "TransferConfig()" applies to S3 client transfers.

session = boto3.Session(
  region_name="us-east-1",
  aws_access_key_id="YOUR_ACCESS_KEY_ID",
  aws_secret_access_key="YOUR_SECRET_ACCESS_KEY",
)

client = session.client("s3",
  config=botocore.client.Config(
    max_pool_connections=50
  )
)

try:
    with open("/my/local/file.ext", "wb") as f:
      client.download_fileobj("MY_S3_BUCKET", object, f, Config=TransferConfig(
          io_chunksize=16 * 1024 * 1024, # 16MB
          max_bandwidth=None,
          max_concurrency=50,
          max_io_queue=1000,
          multipart_chunksize=16 * 1024 * 1024, # 16MB
          multipart_threshold=16 * 1024 * 1024, # 16MB
          use_threads=True
      ))