stringsedescaping

Escape a string for a sed replace pattern


In my bash script I have an external (received from user) string, which I should use in sed pattern.

REPLACE="<funny characters here>"
sed "s/KEYWORD/$REPLACE/g"

How can I escape the $REPLACE string so it would be safely accepted by sed as a literal replacement?

NOTE: The KEYWORD is a dumb substring with no matches etc. It is not supplied by user.


Solution

  • Warning: This does not consider newlines. For a more in-depth answer, see this SO-question instead. (Thanks, Ed Morton & Niklas Peter)

    Note that escaping everything is a bad idea. Sed needs many characters to be escaped to get their special meaning. For example, if you escape a digit in the replacement string, it will turn in to a backreference.

    As Ben Blank said, there are only three characters that need to be escaped in the replacement string (escapes themselves, forward slash for end of statement and & for replace all):

    ESCAPED_REPLACE=$(printf '%s\n' "$REPLACE" | sed -e 's/[\/&]/\\&/g')
    # Now you can use ESCAPED_REPLACE in the original sed statement
    sed "s/KEYWORD/$ESCAPED_REPLACE/g"
    

    If you ever need to escape the KEYWORD string, the following is the one you need:

    sed -e 's/[]\/$*.^[]/\\&/g'

    And can be used by:

    KEYWORD="The Keyword You Need";
    ESCAPED_KEYWORD=$(printf '%s\n' "$KEYWORD" | sed -e 's/[]\/$*.^[]/\\&/g');
    
    # Now you can use it inside the original sed statement to replace text
    sed "s/$ESCAPED_KEYWORD/$ESCAPED_REPLACE/g"
    

    Remember, if you use a character other than / as delimiter, you need replace the slash in the expressions above wih the character you are using. See PeterJCLaw's comment for explanation.

    Edited: Due to some corner cases previously not accounted for, the commands above have changed several times. Check the edit history for details.