[SOLVED] Failed to install Delve for the lambda container on Mac

I used the VS Code AWS Toolkit and SAM to create a basic SAM web app based on Go(I VS Code generate it).
It's some generic template code for the Lambda that reads IP from request and returns a Hello <IP> on the browser.

My end goal is to manage to see line by line debugging on my lambda function in VS Code.
There seem to be some default configurations in my launch.json:

{
    "configurations": [
        {
            "type": "aws-sam",
            "request": "direct-invoke",
            "name": "sunflowers:HelloWorldFunction (go1.x)",
            "invokeTarget": {
                "target": "template",
                "templatePath": "${workspaceFolder}/sunflowers/template.yaml",
                "logicalId": "HelloWorldFunction"
            },
            "lambda": {
                "payload": {},
                "environmentVariables": {},
                "runtime": "go1.x"
            }
        },
        {
            "type": "aws-sam",
            "request": "direct-invoke",
            "name": "API sunflowers:HelloWorldFunction (go1.x)",
            "invokeTarget": {
                "target": "api",
                "templatePath": "${workspaceFolder}/sunflowers/template.yaml",
                "logicalId": "HelloWorldFunction"
            },
            "api": {
                "path": "/hello",
                "httpMethod": "GET",
                "payload": {
                    "json": {}
                }
            },
            "lambda": {
                "runtime": "go1.x"
            }
        }
    ]
}

However on clicking the "Start Debugging" button, I get that error that it can't install delve:

2022-08-03 21:20:46 [ERROR]: log level: info
2022-08-03 21:20:47 [INFO]: Retrieving AWS endpoint data
2022-08-03 21:20:47 [INFO]: OS: Darwin arm64 21.6.0
2022-08-03 21:20:47 [INFO]: Visual Studio Code extension host:  1.69.2
2022-08-03 21:20:47 [INFO]: AWS Toolkit:  1.46.0
2022-08-03 21:20:47 [INFO]: node: 16.13.2
2022-08-03 21:20:47 [INFO]: electron: 18.3.5
2022-08-03 21:21:49 [WARN]: SAM debug: missing AWS credentials (Toolkit is not connected)
2022-08-03 21:21:50 [INFO]: autoconnect: connected: 'profile:default'
2022-08-03 21:21:50 [INFO]: Command: (not started) [/Users/varungawande/Library/Application Support/Code/User/globalStorage/amazonwebservices.aws-toolkit-vscode/debuggers/delve/install-1.9.0.sh ]
2022-08-03 21:21:50 [ERROR]: Failed to cross-compile Delve debugger: Error: spawn Unknown system error -8
    at ChildProcess.spawn (node:internal/child_process:412:11)
    at Object.spawn (node:child_process:718:9)
    at Function.l [as spawn] (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:27:9212)
    at /Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:1734:1495
    at new Promise (<anonymous>)
    at cc.run (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:1734:1099)
    at AH (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:2076:252)
    at _H (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:2075:195)
    at E6.invokeConfig (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:2082:5965)
    at E6.makeAndInvokeConfig (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:2082:1314)
    at E6.resolveDebugConfigurationWithSubstitutedVariables (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:2082:1202) {
  errno: -8,
  code: 'Unknown system error -8',
  syscall: 'spawn'
}
2022-08-03 21:22:27 [INFO]: Command: (not started) [/Users/varungawande/Library/Application Support/Code/User/globalStorage/amazonwebservices.aws-toolkit-vscode/debuggers/delve/install-1.9.0.sh ]
2022-08-03 21:22:27 [ERROR]: Failed to cross-compile Delve debugger: Error: spawn Unknown system error -8
    at ChildProcess.spawn (node:internal/child_process:412:11)
    at Object.spawn (node:child_process:718:9)
    at Function.l [as spawn] (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:27:9212)
    at /Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:1734:1495
    at new Promise (<anonymous>)
    at cc.run (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:1734:1099)
    at AH (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:2076:252)
    at _H (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:2075:195)
    at E6.invokeConfig (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:2082:5965)
    at E6.makeAndInvokeConfig (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:2082:1314)
    at E6.resolveDebugConfigurationWithSubstitutedVariables (/Users/varungawande/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.46.0/dist/src/main.js:2082:1202) {
  errno: -8,
  code: 'Unknown system error -8',
  syscall: 'spawn'
}
2022-08-03 21:25:49 [INFO]: telemetry: sent batch (size=5)

My lambda requires the architecture amd64, while my Mac has the M1 chip, so it has an architecture of arm64. Is the above error due to this architecture mismatch?
I tried running the lambda locally using docker and that worked(using sam local start-api and sam local invoke), but am not sure the above with work as smoothly when using a debugger.