Can't get `sed` to work with text containing escape sequences

I want to be able to use sed (POSIX) to replace some text in the output from a command with a different message that includes escape sequences to allow me to set the colour.

To illustrate the problem.

Create some variables to hold the escape sequences, highlighted and non-highlighted text.

$ _normal="\033[00m";_blue="\033[01;34m"
$ _highlight=" [${_blue}BLUE${_normal}]"
$ _nohighlight=" [BLUE]"

Test that the escape sequences work (the word BLUE is shown in glorious technicolour as expected).

$ echo -e "$_highlight"
 [BLUE]
$

Without any escape sequences the text is displayed normally.

$ echo -e "$_nohighlight"
 [BLUE]
$

I can use sed to replace some of the text with the non-highlighted text.

$ echo ": HIGHLIGHT some more text   " | sed "s/: HIGHLIGHT.\\{1,\\}\$/$_nohighlight/g"
 [BLUE]
$

However, if I try to use the text containing escape sequences it doesn't work!

$ echo ": HIGHLIGHT some more text   " | sed "s/: HIGHLIGHT.\\{1,\\}\$/$_highlight/g"
 [: HIGHLIGHT some more text   33[01;34mBLUE: HIGHLIGHT some more text   33[00m]
$

I tried to work around the problem by replacing the text with a newline character instead, but can't get that to work either. (Bash: Unable to replace bold text with plain text in a string using Sed)

Solution

The problem is that you are relying on echo -e to perform processing of the \033 escape codes for you, so if you don't pass the data through that command (or something else that does the same processing) then what reaches the terminal doesn't contain a proper escape code.

Bash provides an alternative: a (sub)string quoted with $'...' delimiters is subject to escape processing by the shell, according to ANSI C escape codes, including octal escapes such as the \033 in your example data. Using this mechanism, you can let the shell do the escape processing. Like so, for example:

$ _normal=$'\033[00m'
$ _blue=$'\033[01;34m'
$ _highlight=" [${_blue}BLUE${_normal}]"
$ echo "$_highlight"
 [BLUE]
$ echo ": HIGHLIGHT some more text   " | sed 's/: HIGHLIGHT.\{1,\}$/'"$_highlight/g"
 [BLUE]
$

Note that that uses just echo, no -e, and that the escape codes pass through sed unmolested.

Additional Note

Although sed is not sensitive to escape codes in the data stream it processes, ANSI terminal control codes contain the [ character, which is meaningful in sed expressions as a regex metacharacter. It takes some care, therefore, if you want to select lines based on terminal control sequences, or if you want to use the s command to manipulate terminal control sequences or text associated with them, such that a control sequence needs to be matched by a pattern.