How to make OpenHands (Running on Docker on macOS) to Work with AWS Bedrock?
I'm setting up OpenHands with AWS Bedrock on macOS using Docker, but encountering connection issues related to the Docker client and server API version. While some commands inside the container work, the main application fails with the following errors:
Docker Run Command:
docker run -it --pull=always \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE="docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik" \
-e AWS_ACCESS_KEY_ID="YOUR_ACCESS_KEY_ID" \
-e AWS_SECRET_ACCESS_KEY="YOUR_SECRET_ACCESS_KEY" \
-e AWS_REGION="us-east-1" \
-e LLM_PROVIDER="bedrock" \
-e MODEL_ID="anthropic.claude-v2" \
-e BASE_URL="https://bedrock.us-east-1.amazonaws.com" \
-p 3000:3000 \
--add-host=host.docker.internal:host-gateway \
--name openhands-app-bedrock \
docker.all-hands.dev/all-hands-ai/openhands:latest
Successfully ran commands inside the container:
docker exec -it openhands-app-bedrock /bin/bash and inside ipython:
from litellm import completion
response = completion(model="anthropic.claude-v2", messages=[{"role": "user", "content": "Hello, how are you?"}])
print(response.choices[0].message.content)
# Output: I'm doing well, thanks for asking!
Error Details:
Starting OpenHands...
Running OpenHands as root
INFO: Started server process [9]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:3000 (Press CTRL+C to quit)
INFO: 192.168.65.1:57134 - "GET / HTTP/1.1" 200 OK
INFO: 192.168.65.1:57134 - "GET /locales/en/translation.json HTTP/1.1" 200 OK
INFO: 192.168.65.1:57134 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO: 192.168.65.1:57134 - "GET /config.json HTTP/1.1" 200 OK
INFO: 192.168.65.1:57135 - "GET /config.json HTTP/1.1" 200 OK
INFO: 192.168.65.1:57135 - "GET /config.json HTTP/1.1" 200 OK
INFO: 192.168.65.1:57135 - "GET /favicon.ico HTTP/1.1" 200 OK
18:19:26 - openhands:INFO: github.py:14 - Initializing UserVerifier
18:19:26 - openhands:INFO: github.py:27 - GITHUB_USER_LIST_FILE not configured
18:19:26 - openhands:INFO: github.py:48 - GITHUB_USERS_SHEET_ID not configured
18:19:26 - openhands:INFO: github.py:85 - No user verification sources configured - allowing all users
INFO: ('192.168.65.1', 57136) - "WebSocket /ws" [accepted]
18:19:26 - openhands:ERROR: auth.py:27 - Invalid token
INFO: 192.168.65.1:57135 - "GET /api/options/models HTTP/1.1" 200 OK
INFO: connection open
INFO: 192.168.65.1:57134 - "GET /api/options/agents HTTP/1.1" 200 OK
INFO: 192.168.65.1:57137 - "GET /api/options/security-analyzers HTTP/1.1" 200 OK
INFO: connection closed
INFO: 192.168.65.1:57137 - "GET /assets/end-session-CpBsKLW_.js HTTP/1.1" 200 OK
INFO: 192.168.65.1:57137 - "GET /config.json HTTP/1.1" 200 OK
INFO: 192.168.65.1:57134 - "GET /config.json HTTP/1.1" 200 OK
INFO: 192.168.65.1:57134 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO: 192.168.65.1:57134 - "GET /config.json HTTP/1.1" 200 OK
INFO: 192.168.65.1:57134 - "GET /favicon.ico HTTP/1.1" 200 OK
18:19:28 - openhands:INFO: github.py:14 - Initializing UserVerifier
18:19:28 - openhands:INFO: github.py:27 - GITHUB_USER_LIST_FILE not configured
18:19:28 - openhands:INFO: github.py:48 - GITHUB_USERS_SHEET_ID not configured
18:19:28 - openhands:INFO: github.py:85 - No user verification sources configured - allowing all users
INFO: ('192.168.65.1', 57138) - "WebSocket /ws" [accepted]
18:19:28 - openhands:INFO: listen.py:336 - New session: aee02a3d-97c4-4c23-9df5-61094a47a0bf
INFO: connection open
18:19:28 - openhands:WARNING: codeact_agent.py:101 - Function calling not supported for model anthropic.claude-v2. Disabling function calling.
INFO: 192.168.65.1:57134 - "GET /config.json HTTP/1.1" 200 OK
18:19:28 - openhands:INFO: base.py:98 - [runtime aee02a3d-97c4-4c23-9df5-61094a47a0bf] Starting runtime with image: docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik
18:19:28 - openhands:INFO: base.py:98 - [runtime aee02a3d-97c4-4c23-9df5-61094a47a0bf] Container started: openhands-runtime-aee02a3d-97c4-4c23-9df5-61094a47a0bf
18:19:28 - openhands:INFO: base.py:98 - [runtime aee02a3d-97c4-4c23-9df5-61094a47a0bf] Waiting for client to become ready at http://host.docker.internal:36410...
18:19:42 - openhands:INFO: base.py:98 - [runtime aee02a3d-97c4-4c23-9df5-61094a47a0bf] Runtime is ready.
18:19:42 - openhands:WARNING: state.py:118 - Failed to restore state from session: sessions/aee02a3d-97c4-4c23-9df5-61094a47a0bf/agent_state.pkl
18:19:42 - openhands:INFO: agent_controller.py:135 - [Agent Controller aee02a3d-97c4-4c23-9df5-61094a47a0bf] Starting step loop...
18:19:42 - openhands:INFO: agent_controller.py:135 - [Agent Controller aee02a3d-97c4-4c23-9df5-61094a47a0bf] Setting agent(CodeActAgent) state from AgentState.LOADING to AgentState.INIT
18:19:42 - openhands:INFO: agent_controller.py:135 - [Agent Controller aee02a3d-97c4-4c23-9df5-61094a47a0bf] Setting agent(CodeActAgent) state from AgentState.INIT to AgentState.RUNNING
18:19:42 - openhands:INFO: github.py:14 - Initializing UserVerifier
18:19:42 - openhands:INFO: github.py:27 - GITHUB_USER_LIST_FILE not configured
18:19:42 - openhands:INFO: github.py:48 - GITHUB_USERS_SHEET_ID not configured
18:19:42 - openhands:INFO: github.py:85 - No user verification sources configured - allowing all users
INFO: 192.168.65.1:57331 - "GET /config.json HTTP/1.1" 200 OK
INFO: 192.168.65.1:57337 - "GET /config.json HTTP/1.1" 200 OK
INFO: 192.168.65.1:57330 - "GET /api/list-files HTTP/1.1" 200 OK
18:19:42 - openhands:INFO: github.py:14 - Initializing UserVerifier
18:19:42 - openhands:INFO: github.py:27 - GITHUB_USER_LIST_FILE not configured
18:19:42 - openhands:INFO: github.py:48 - GITHUB_USERS_SHEET_ID not configured
18:19:42 - openhands:INFO: github.py:85 - No user verification sources configured - allowing all users
INFO: 192.168.65.1:57337 - "GET /api/list-files HTTP/1.1" 200 OK
18:19:43 - openhands:INFO: github.py:14 - Initializing UserVerifier
18:19:43 - openhands:INFO: github.py:27 - GITHUB_USER_LIST_FILE not configured
18:19:43 - openhands:INFO: github.py:48 - GITHUB_USERS_SHEET_ID not configured
18:19:43 - openhands:INFO: github.py:85 - No user verification sources configured - allowing all users
INFO: 192.168.65.1:57330 - "GET /api/list-files HTTP/1.1" 200 OK
==============
[Agent Controller aee02a3d-97c4-4c23-9df5-61094a47a0bf] LEVEL 0 LOCAL STEP 0 GLOBAL STEP 0
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.
18:19:44 - openhands:ERROR: retry_mixin.py:47 - litellm.APIConnectionError: 'output'
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 2564, in completion
response = bedrock_converse_chat_completion.completion(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_handler.py", line 421, in completion
return litellm.AmazonConverseConfig()._transform_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_transformation.py", line 390, in _transform_response
message: Optional[MessageBlock] = completion_response["output"]["message"]
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'output'
. Attempt #1 | You can customize retry values in the configuration.
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.
18:19:59 - openhands:ERROR: retry_mixin.py:47 - litellm.APIConnectionError: 'output'
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 2564, in completion
response = bedrock_converse_chat_completion.completion(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_handler.py", line 421, in completion
return litellm.AmazonConverseConfig()._transform_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_transformation.py", line 390, in _transform_response
message: Optional[MessageBlock] = completion_response["output"]["message"]
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'output'
. Attempt #2 | You can customize retry values in the configuration.
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.
18:20:14 - openhands:ERROR: retry_mixin.py:47 - litellm.APIConnectionError: 'output'
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 2564, in completion
response = bedrock_converse_chat_completion.completion(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_handler.py", line 421, in completion
return litellm.AmazonConverseConfig()._transform_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_transformation.py", line 390, in _transform_response
message: Optional[MessageBlock] = completion_response["output"]["message"]
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'output'
. Attempt #3 | You can customize retry values in the configuration.
Solution
To resolve the connection issue with AWS Bedrock, try running the following command:
docker run -it --pull=always \
-e LITELLM_LOG='DEBUG' \
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
-e AWS_ACCESS_KEY_ID='<YOUR_ACCESS_KEY>' \
-e AWS_SECRET_ACCESS_KEY='<YOUR_SECRET_KEY>' \
-e AWS_REGION_NAME='us-east-1' \
-v /var/run/docker.sock:/var/run/docker.sock \
-v ~/.aws:/root/.aws:ro \
-p 3000:3000 \
--add-host host.docker.internal:host-gateway \
--name openhands-app \
docker.all-hands.dev/all-hands-ai/openhands:0.12
For more, refer to LiteLLM’s Bedrock provider docs: https://docs.litellm.ai/docs/providers/bedrock
And make sure the configuration is something like the following:
- Convert Categorical codes to Categorical values
- Python-Sphinx: "inherit" method documentation from superclass
- Getting Home Directory with pathlib
- How to redirect the user back to the home page using FastAPI, after submitting an HTML form?
- No FileSystem for scheme: s3 with pyspark
- How to add a state to a Selection field and show it ordered in the statusbar in Odoo 11?
- Breaking long method chains into multiple lines in Python
- Enable ruff rules only on specific files
- Filter by time in DRF
- Plotting lambda functions in Python and mpmath.plot
- How to remove Empty Rows from a list in Python 3.5
- matplotlib 3D scatter plot alpha varies when viewing different angles
- How to write very long string that conforms with PEP8 and prevent E501
- how to avoid bot detection on websites using selenium python
- Python mock to create a fake object return a dictionary when any of its attributes are used
- Polars vs. Pandas: size and speed difference
- How to mock.patch a class imported in another module
- Python - error cannot determine truth value of Relational (Newton-Raphson)
- ProcessPoolExecutor logging fails to log inside function on Windows but not on Unix / Mac
- SQLAlchemy ORM Insert or Update when importing from JSON
- django managers vs proxy models
- Pytroch clamp for complex values
- For every identifier select only rows with largest order column
- truth value for Expr is ambiguous in with_columns ternary expansion on dates
- Remove equal characters from two python strings
- Python pyad module can't set UPN
- Macro VS Micro VS Weighted VS Samples F1 Score
- Printing a Tree data structure in Python
- How to fix/reset decreasing timestamps while preserving gaps in time-series data for CNN training?
- Test that module is NOT imported