Here is my test.env
RABBITMQ_HOST=127.0.0.1
RABBITMQ_PASS=1234
And I want to use test.sh
to replace the value in test.env
to :
RABBITMQ_HOST=rabbitmq1
RABBITMQ_PASS=12345
here is my test.sh
#!/bin/bash
echo "hello world"
RABBITMQ_HOST=rabbitmq1
RABBITMQ_PASS=12345
Deploy_path="./config/test.env"
sed -i 's/RABBITMQ_HOST=.*/RABBITMQ_HOST='$RABBITMQ_HOST'/' $Deploy_path
sed -i 's/RABBITMQ_PASS=.*/RABBITMQ_PASS='$RABBITMQ_HOST'/' $Deploy_path
But I have error
sed: 1: "./config/test.env": invalid command code .
sed: 1: "./config/test.env": invalid command code .
How can I fix it?
tl;dr:
With BSD Sed, such as also found on macOS, you must use -i ''
instead of just -i
(for not creating a backup file) to make your commands work; e.g.:
sed -i '' 's/RABBITMQ_HOST=.*/RABBITMQ_HOST='"$RABBITMQ_HOST"'/' "$Deploy_path"
To make your command work with both GNU and BSD Sed, specify a nonempty option-argument (which creates a backup) and attach it directly to -i
:
sed -i'.bak' 's/RABBITMQ_HOST=.*/RABBITMQ_HOST='"$RABBITMQ_HOST"'/' "$Deploy_path" &&
rm "$Deploy_path.bak" # remove unneeded backup copy
Background information, (more) portable solutions, and refinement of your commands can be found below.
It sounds like you're using BSD/macOS sed
, whose -i
option requires an option-argument that specifies the suffix of the backup file to create.
Therefore, it is your sed
script that (against your expectations) is interpreted as -i
's option-argument (the backup suffix), and your input filename is interpreted as the script, which obviously fails.
By contrast, your commands use GNU sed
syntax, where -i
can be used by itself to indicate that no backup file of the input file to updated in-place is to be kept.
The equivalent BSD sed
option is -i ''
- note the technical need to use a separate argument to specify the option-argument ''
, because it is the empty string (if you used -i''
, the shell would simply strip the ''
before sed
ever sees it: -i''
is effectively the same as just -i
).
Sadly, this then won't work with GNU sed
, because it only recognizes the option-argument when directly attached to -i
, and would interpret the separate ''
as a separate argument, namely as the script.
This difference in behavior stems from a fundamentally differing design decision behind the implementation of the -i
option and it probably won't go away for reasons of backward compatibility.[1]
If you do not want a backup file created, there is no single -i
syntax that works for both BSD and GNU sed
.
There are four basic options:
(a) If you know that you'll only be using either GNU or BSD sed
, construct the -i
option accordingly: -i
for GNU sed
, -i ''
for BSD sed
.
(b) Specify a nonempty suffix as -i
's option-argument, which, if you attach it directly to the -i
option, works with both implementations; e.g., -i'.bak'
. While this invariably creates a backup file with suffix .bak
, you can just delete it afterward.
(c) Determine at runtime which sed
implementation you're dealing with and construct the -i
option accordingly.
(d) omit -i
(which is not POSIX-compliant) altogether, and use a temporary file that replaces the original on success: sed '...' "$Deploy_path" > tmp.out && mv tmp.out "$Deploy_path"
.
Note that this is in essence what -i
does behind the scenes, which can have unexpected side effects, notably an input file that is a symlink getting replaced with a regular file; -i
, does, however, preserve certain attributes of the original file: see the lower half of this answer of mine.
Here's a bash
implementation of (c) that also streamlines the original code (single sed
invocation with 2 substitutions) and makes it more robust (variables are double-quoted):
#!/bin/bash
RABBITMQ_HOST='rabbitmq1'
RABBITMQ_PASS='12345'
Deploy_path="test.env"
# Construct the Sed-implementation-specific -i option-argument.
# Caveat: The assumption is that if the `sed` is not GNU Sed, it is BSD Sed,
# but there are Sed implementations that don't support -i at all,
# because, as Steven Penny points out, -i is not part of POSIX.
suffixArg=()
sed --version 2>/dev/null | grep -q GNU || suffixArg=( '' )
sed -i "${suffixArg[@]}" '
s/^\(RABBITMQ_HOST\)=.*/\1='"$RABBITMQ_HOST"'/
s/^\(RABBITMQ_PASS\)=.*/\1='"$RABBITMQ_PASS"'/
' "$Deploy_path"
Note that with the specific values defined above for $RABBITMQ_HOST
and $RABBITMQ_PASS
, it is safe to splice them directly into the sed
script, but if the values contained instances of &
, /
, \
, or newlines, prior escaping would be required so as not to break the sed
command.
See this answer of mine for how to perform generic pre-escaping, but you may also consider other tools at that point, such as awk
and perl
.
[1] GNU Sed considers the option-argument to -i optional, whereas BSD Sed considers it mandatory, which is also reflected in the syntax specs. in the respective man
pages: GNU Sed: -i[SUFFIX]
vs. BSD Sed -i extension
.