How to change cursor shape depending on VI mode in Bash?

I have the following line in my .bashrc:

set -o vi

And I want my cursor to have a pipe shape when I am in insert mode and a block shape when I am in command mode, like what I would have in Vim if I placed the following in my .vimrc:

let &t_SI = "\e[6 q"
let &t_SR = "\e[4 q"
let &t_EI = "\e[2 q"

Except in this case I want to have the equivalent behavior on the command line.

I found a partial answer to my question here - https://unix.stackexchange.com/questions/22527/change-cursor-shape-or-color-to-indicate-vi-mode-in-bash - written by @gogolb.

Here is the answer, copied:

#!/bin/bash
# Script "kmtest.sh"

TEST=`bind -v | awk '/keymap/ {print $NF}'`
if [ "$TEST" = 'vi-insert' ]; then
    echo -ne "\033]12;Green\007"
else
    echo -ne "\033]12;Red\007"
fi

export PS1="\u@\h \$(kmtest.sh)> "

Unfortunately, though, as explained in the answer, the example script only changes the cursor shape after a carriage return, whereas, what I want is for the cursor shape to change when I hit <Esc> (i.e. when I change mode).

I am on Linux running the native terminal app, with Bash 4.4.7 and my $TERM variable set to xterm-256color. Also, I do not know if tmux has any effect on what I am asking for, but I, ideally, would like the solution to work both within and exterior to tmux sessions.

SOLUTION

I ended up discovering the answer to this question myself, which I describe here in another question I posted:

How to correctly link patched GNU readline library to all existing programs?

Don't worry, the solution does not require any patching. ;)

Solution

SOLUTION:

I am posting my answer to my own question here as recommended.

This solution works for Bash 4.4+, since, starting with that version of Bash, version 7.0 of the GNU readline library is used, which includes the necessary additions of the vi-cmd-mode-string and vi-ins-mode-string variables.

These variables can be set as follows in your .inputrc file in order to achieve the functionality I described above:

set show-mode-in-prompt on
set vi-cmd-mode-string "\1\e[2 q\2"
set vi-ins-mode-string "\1\e[6 q\2"

EXPLANATION:

For those who are actually interested in how the above solution works.

These two variables, vi-cmd-mode-string and vi-ins-mode-string, are printed to your terminal along with the command prompt in order to provide a sort of visual indicator as to which mode you are currently in (i.e. command mode vs. insert mode).

The defaults for these two variables are "(cmd)" and "(ins)" for command and insert modes, respectively. So if you were to just leave them as the defaults and had a command prompt of, say, PS1='>>>', then your prompt would look like the following:

Command mode:
```
  (cmd) >>>
```
Insert mode:
```
  (ins) >>>
```

According to the man-page for readline (see below), you can also specify non-printable characters, such as terminal control sequences, by embedding the sequences between the \1 and \2 escape characters.

vi-cmd-mode-string ((cmd))
       This  string  is  displayed immediately before the last line of the primary prompt when vi editing mode is active and in command mode.  The value is expanded like a key binding, so the
       standard set of meta- and control prefixes and backslash escape sequences is available.  Use the \1 and \2 escapes to begin and end sequences of non-printing characters, which  can  be
       used to embed a terminal control sequence into the mode string.
vi-ins-mode-string ((ins))
       This  string is displayed immediately before the last line of the primary prompt when vi editing mode is active and in insertion mode.  The value is expanded like a key binding, so the
       standard set of meta- and control prefixes and backslash escape sequences is available.  Use the \1 and \2 escapes to begin and end sequences of non-printing characters, which  can  be
       used to embed a terminal control sequence into the mode string.

Therefore, in my above solution, I am embedding the terminal control sequences, \e[2 q (make the cursor a vertical bar) and \e[6 q (make the cursor a pipe), between these \1 and \2 escape characters, resulting in my cursor having the shape of a vertical bar while in command mode and a pipe shape while in insert mode.