In my company when we SSH to our AWS EC2 instances we are required to use the aws CLI session-manager plugin for auth. Using this SSH config snippet works:
Host my-aws-host
ProxyCommand bash -c "aws ssm start-session --target 'i-0abc123def456hij' \
--document-name AWS-StartSSHSession --parameters 'portNumber=22' \
--region us-west-1 --profile MAIN"
However, when the EC2 instance is relaunched, which happens semi-regularly, the 'target' instance ID changes. When this happens, all users need to update their SSH config with the new ID. We don't have any sort of DNS that resolves these instances to a static hostname unfortunately, and so would need to somehow publish the new instance ID to all interested users.
So instead I wrote a bash script (ssh_proxy_command.sh
) that first queries our AWS account to grab the current instance ID based on a known tag value, and use that for the target - here's a cut-down version:
#!/bin/bash
INSTANCE_ID=$(aws ec2 describe-instances --region us-west-1 \
--filters Name=tag:Name,Values=my-server-nametag* \
--query "Reservations[*].Instances[*].{Instance:InstanceId}" --output text)
aws ssm start-session --target $INSTANCE_ID --document-name AWS-StartSSHSession --parameters 'portNumber=22' --region us-west-1 --profile MAIN
Now the SSH config looks like
Host my-aws-host
ProxyCommand bash -c "/path/to/my/ssh_proxy_command.sh %h"
This has been working fine. However, we have just started running multiple instances built from the same base image (AMI), and which use the same tags, etc. so the given describe-instances query now returns multiple instance IDs. So I tried wrapping the output returned by the query in a bash select
loop, thinking I could offer the user a list of instance IDs and let them choose the one they want. This works when running the script directly, but not when it's used as the ProxyCommand. In the latter case when it reaches the select
statement it prints out the options as expected, but doesn't wait for the user input - it just continues straight to the end of the script with an empty $INSTANCE_ID variable, which makes the aws ssm
command fail.
I'm guessing this is a side-effect of the way SSH runs its ProxyCommands — from the ssh_config
man page:
[the proxy command] is executed using the user's shell ‘exec’ directive [...]
I'm hoping I can find a way around this problem while still using SSH config and ProxyCommand, rather than resorting to a complete stand-alone wrapper around the ssh
executable and requiring everyone use that. Any suggestions gratefully accepted...
I faced the same issue trying to achieve exactly what you're trying to do here.
I solved by writing and reading to/from tty
as it seems the only stream available in the spawned exec
command is /dev/stderr
(select
command outputs to stderr
). Following, the solution I adopted:
~/.ssh/config:
ProxyCommand sh ~/ssm-node-selection.sh <PROFILE>
~/ssm-node-selection.sh:
PROFILE=$1
TTY="/dev/tty$(ps ax | grep "^$$" | awk '{ print $2 }')"
EC2_INSTANCE_IDS=$(aws ec2 describe-instances --profile $PROFILE --query "Reservations[].Instances[].InstanceId" --filters "Name=tag:Name,Values=<INSTANCE_NAME>" | jq '. | join(" ")' | tr -d '"')
echo $EC2_INSTANCE_IDS > $TTY
echo "Choose an instance: " > $TTY
read INSTANCE_ID < $TTY
aws ec2-instance-connect send-ssh-public-key --profile $PROFILE --instance-id $INSTANCE_ID --instance-os-user <USER> --ssh-public-key file://<PUBLIC_KEY> && aws ssm start-session --target $INSTANCE_ID --profile $PROFILE --document-name AWS-StartSSHSession