aws-cdkamazon-bedrock

Invocation of model ID anthropic.claude-3-5-sonnet-20241022-v2:0 with on-demand throughput isn’t supported Bedrock error

I am using python cdk to launch a Bedrock Agent which uses anthropic.claude-3-5-sonnet-20241022-v2:0 as a foundation model.

        support_agent = bedrock.CfnAgent(self, "HorizonIQSupportAgent",
        agent_name="support-agent",
        agent_resource_role_arn=agent_role.attr_arn,
        auto_prepare=True,
        description="support-agent",
        foundation_model=foundation_model_name,
        idle_session_ttl_in_seconds=600,
        instruction=prompt,
        knowledge_bases=[bedrock.CfnAgent.AgentKnowledgeBaseProperty(
            description=kb_description,
            knowledge_base_id=knowledge_base.attr_knowledge_base_id,
        )]
    )

this works fine and the agent gets deployed. However, I am not able to test it on the console and I get below error.

enter image description here

Invocation of model ID anthropic.claude-3-5-sonnet-20241022-v2:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model

I have to manually edit the model and select it again with an inference profile to make it work. How can i do it automatically using python cdk ?

before

enter image description here

after manual editing

enter image description here

Solution

  • Apparently, claude sonnet 3.5 V2 is only available in us-west-2. So in order to use in us-east-1, you have to use the inference profile is as the model id.

    So in my cdk, I replaced model from - anthropic.claude-3-5-sonnet-20241022-v2:0 to us.anthropic.claude-3-5-sonnet-20241022-v2:0

    You can find this in Inference and Assessment > Cross-region inference