bashawksed

Replace after searched string with a variable (containing slashes) in sed/awk


Tools

Attempted behaviour

Input

.colour_list;

#000000
#000001
#000002
#000004
#000005
#000006
#000007
#000008
#000009
#00000A
#00000B
#00000C
#00000D
#00000E
#00000F

Process (script)

tst.sh;

#!/bin/bash

value=9
transparency=FF

bc=$(head -n${value} ~/.colour_list | 
    tr -d '#' | 
    sed 's/.\{2\}/&\//g;s/.$//' | 
    awk '{print "rgba:" $0 transparency}' transparency=$transparency | 
    sed 's/.\{16\}/&,/g')

sed -ri "s|^(bar_colour             \s*=\s*).*|\1$bc|" ~/test.conf

echo $bc

Output

Output:

bash output;

sed: -e expression #1, char 52: unterminated `s' command
rgba:00/00/00/FF rgba:00/00/01/FF rgba:00/00/02/FF rgba:00/00/04/FF rgba:00/00/05/FF rgba:00/00/06/FF rgba:00/00/07/FF rgba:00/00/08/FF rgba:00/00/09/FF

~/test.conf not changed;

...
bar_colour              = test
...

Expected output:

bash output;

rgba:00/00/00/FF, rgba:00/00/01/FF, rgba:00/00/02/FF, rgba:00/00/04/FF, rgba:00/00/05/FF, rgba:00/00/06/FF, rgba:00/00/07/FF, rgba:00/00/08/FF, rgba:00/00/09/FF

~/test.conf changed;

...
bar_colour              = rgba:00/00/00/FF, rgba:00/00/01/FF, rgba:00/00/02/FF, rgba:00/00/04/FF, rgba:00/00/05/FF, rgba:00/00/06/FF, rgba:00/00/07/FF, rgba:00/00/08/FF, rgba:00/00/09/FF
...

Issues

I have two issues.

  1. The $bc assignment (see sed 's/.\{2\}/&\//g;s/.$//') removes all commas, where I would only like to remove the final comma (i.e. the final character) of the entire string. Please note, I do require the $bc assignment (see here).

  2. Upon finding the string, colour = , I want to place the value of $bc after it but I instead get an error (see above). Please note that I would like to retain the gap between colour and =, which is a mixture of tabs and spaces.

Methods tried

I have tried previous answers (1, 2, 3, 4) but none of them seem to work and invariably return as above.

I am guessing for the awk error at least, this is because the variable I am trying to replace with contains slashes, but I am not sure why sed is throwing an error. I require the slashes as the expected format is as per XQueryColor specifications.

I have placed the script into ShellCheck. Apart from suggesting $bc be wrapped in double quotes at the final line (which is not necessary for the script's functioning), it reports No issues detected!.

Changed script to the following:

value=9
transparency=FF

bc=$(head -n${value} ~/.colour_list | 
    tr -d '#' | 
    sed 's@.\{2\}@&\/@g;s@.$@@' | 
    awk '{print "rgba:" $0 transparency}' transparency=$transparency | 
    sed 's@.\{16\}@&,@g')

sed -ri "s|^(bar_colour             \s*=\s*).*|\1$bc|" ~/test.conf

echo $bc

This also provided the same errors as per the Output section, unfortunately.


Solution

  • $ cat test.conf
    ...
    bar_colour             = foo
    bar_colour              = test
    bar_colour               = bar
    ...
    

    $ cat tst.sh
    #!/usr/bin/env bash
    
    tmp=$(mktemp) || exit
    trap 'rm -f "$tmp"; exit' EXIT
    
    awk '
        NR == FNR {
            sub(/^#/,"")
            gsub(/../,"&/"); sub("/$","")
            bc = bc sprintf("%srgba: %s/2F", (NR>1 ? ", " : ""), $0)
            if ( NR == 9 ) {
                nextfile
            }
            next
        }
        match($0,/^bar_colour              = /) {
            $0 = substr($0,1,RLENGTH) bc
        }
        { print }
    ' .colour_list test.conf > "$tmp" &&
    mv "$tmp" test.conf
    

    $ ./tst.sh
    

    $ cat test.conf
    ...
    bar_colour             = foo
    bar_colour              = rgba: 00/00/00/2F, rgba: 00/00/01/2F, rgba: 00/00/02/2F, rgba: 00/00/04/2F, rgba: 00/00/05/2F, rgba: 00/00/06/2F, rgba: 00/00/07/2F, rgba: 00/00/08/2F, rgba: 00/00/09/2F
    bar_colour               = bar
    ...
    

    How we got there...

    Stepping through your "Attempted behaviour" list one at a time:

    Grab the first 9 colours from ~/.colour_list which has the colours printed with a newline between each one

    awk '
        { print }
        NR == 9 { exit }
    ' .colour_list
    #000000
    #000001
    #000002
    #000004
    #000005
    #000006
    #000007
    #000008
    #000009
    

    Remove the leading # from each colour

    awk '
        {
            sub(/^#/,"")
            print
        }
        NR == 9 { exit }
    ' .colour_list
    000000
    000001
    000002
    000004
    000005
    000006
    000007
    000008
    000009
    

    Place a / every 2nd character; FF/FF/FF/

    awk '
        {
            sub(/^#/,"")
            gsub(/../,"&/"); sub("/$","")
            print
        }
        NR == 9 { exit }
    ' .colour_list
    00/00/00
    00/00/01
    00/00/02
    00/00/04
    00/00/05
    00/00/06
    00/00/07
    00/00/08
    00/00/09
    

    Append each colour with a transparency value; FF/FF/FF/2F

    awk '
        {
            sub(/^#/,"")
            gsub(/../,"&/"); sub("/$","")
            print $0 "/2F"
        }
        NR == 9 { exit }
    ' .colour_list
    00/00/00/2F
    00/00/01/2F
    00/00/02/2F
    00/00/04/2F
    00/00/05/2F
    00/00/06/2F
    00/00/07/2F
    00/00/08/2F
    00/00/09/2F
    

    Prepend each colour with the string rgba:; rgba: FF/FF/FF/2F

    awk '
        {
            sub(/^#/,"")
            gsub(/../,"&/"); sub("/$","")
            print "rgba:", $0 "/2F"
        }
        NR == 9 { exit }
    ' .colour_list
    rgba: 00/00/00/2F
    rgba: 00/00/01/2F
    rgba: 00/00/02/2F
    rgba: 00/00/04/2F
    rgba: 00/00/05/2F
    rgba: 00/00/06/2F
    rgba: 00/00/07/2F
    rgba: 00/00/08/2F
    rgba: 00/00/09/2F
    

    Place a , every 16 characters (except for the last character) to create a comma separated list; rgba: FF/FF/FF/2F, rgba: 2F/AA/66/2F ...

    awk '
        {
            sub(/^#/,"")
            gsub(/../,"&/"); sub("/$","")
            printf "%srgba: %s/2F", (NR>1 ? ", " : ""), $0
        }
        NR == 9 { print ""; exit }
    ' .colour_list
    rgba: 00/00/00/2F, rgba: 00/00/01/2F, rgba: 00/00/02/2F, rgba: 00/00/04/2F, rgba: 00/00/05/2F, rgba: 00/00/06/2F, rgba: 00/00/07/2F, rgba: 00/00/08/2F, rgba: 00/00/09/2F
    

    Place this entire comma separated list into a variable bc

    bc=$( awk '
        {
            sub(/^#/,"")
            gsub(/../,"&/"); sub("/$","")
            printf "%srgba: %s/2F", (NR>1 ? ", " : ""), $0
        }
        NR == 9 { print ""; exit }
    ' .colour_list )
    

    Place the variable bc after the exact string match bar_color = (including spaces) in the file test.conf

    $ cat tst.sh
    #!/usr/bin/env bash
    
    bc=$( awk '
        {
            sub(/^#/,"")
            gsub(/../,"&/"); sub("/$","")
            printf "%srgba: %s/2F", (NR>1 ? ", " : ""), $0
        }
        NR == 9 { print ""; exit }
    ' .colour_list )
    printf 'bar_colour              = %s\n' "$bc" > test.conf
    
    cat test.conf
    

    $ ./tst.sh
    bar_colour              = rgba: 00/00/00/2F, rgba: 00/00/01/2F, rgba: 00/00/02/2F, rgba: 00/00/04/2F, rgba: 00/00/05/2F, rgba: 00/00/06/2F, rgba: 00/00/07/2F, rgba: 00/00/08/2F, rgba: 00/00/09/2F
    

    It's not clear why you want a shell variable bc or to format the output for test.conf outside of awk instead of just:

    awk '
        {
            sub(/^#/,"")
            gsub(/../,"&/"); sub("/$","")
            bc = bc sprintf("%srgba: %s/2F", (NR>1 ? ", " : ""), $0)
        }
        NR == 9 {
            printf "bar_colour              = %s\n", bc
            exit
        }
    ' .colour_list > test.conf
    

    $ cat test.conf
    bar_colour              = rgba: 00/00/00/2F, rgba: 00/00/01/2F, rgba: 00/00/02/2F, rgba: 00/00/04/2F, rgba: 00/00/05/2F, rgba: 00/00/06/2F, rgba: 00/00/07/2F, rgba: 00/00/08/2F, rgba: 00/00/09/2F