bashshelltext-manipulation

Replace multiline section of a file with a multiline string, both containing special characters and escape sequences

I have a file which has a section like the following, only one key/value pair is provided but there could be N number:

    "Section Name"
    {
        "Key"       "{\"foo\":[\"bar\"],\"one\":\"two\"}"
        "AnotherValue"      "I am a string"
    }

Using Bash, I am able to extract, parse, and manipulate the block I want, and can store the manipulated value in a string which looks like this:

    "Section Name"
    {
        "Key"       "{\"changed\":[\"value\"],\"three\":\"two\"}"
        "AnotherValue"      "Updated string"
    }

The values in the section are arbitrary, though they will always be enclosed in quotes. There are many JSONified strings, but many plaintext strings as shown as well. This is just one example, the contents of the block could be emptied entirely, contain entirely different values, and so on. The block may not contain any special characters, or may include key/value pays with the value as simply "" (though it will always, at least, be empty quotes).

I need to update the source file's original block with the manipulated block, using Bash and standard GNU Shell utils, exactly as shown, with the exact escape sequences. It should ideally go in the same position in the file as the original block, just replacing it.

So if the file reads like this:

    "Blank Section 1"
    {
    }
    "Section Name"
    {
        "Key"       "{\"foo\":[\"bar\"],\"one\":\"two\"}"
        "AnotherValue"      "I am a string"
    }
    "Blank Section 2"
    {
    }

The updated file should read like this:

    "Blank Section 1"
    {
    }
    "Section Name"
    {
        "Key"       "{\"changed\":[\"value\"],\"three\":\"two\"}"
        "AnotherValue"      "Updated string"
    }
    "Blank Section 2"
    {
    }

I have the source block and manipulated block both stored as variables, and if necessary it would also be acceptable to use the start and end lines of the updated section as patterns to replace between.

I have tried using sed and awk to accomplish this separately with no luck, various solutions work up to the point of trying to use values which have these escaped special characters. I can sort of get sed to work if I mangle the input strings to be escaped, but I need to preserve the entire file contents as-is and only need to replace one block with another.

I can assume that the start and end patterns for the block won't change when the block is manipulated, the block contents are the only thing that will change, and I can assume they will always follow that format: one tab indented from the base indentation amount, key surrounded by quotes, two tabs, then a value surrounded by quotes with any arbitrary value (and stringified JSON blobs or similar will be escaped).

To be clear, I am not asking how to manipulate the block. I have already done this. I simply need a way to replace the original block with the updated block, preserving all escape characters as shown. I have the exact source and exact manipulated block already stored in variables, I just need to replace the source block with the updated block.

It would be acceptable to replace based on line numbers of the original file, such as removing everything between line X and Y, and then inserting the updated block on Line X. Likewise, taking the start and end lines as patterns to replace between is also fully acceptable. So long as the original block is overwritten with the new block, the escape sequences are preserved exactly as shown (if present at all), and the rest of the file content remains present but unaffected, and all of this is accomplished with standard Bash and GNU utilities, any solution works for me.

The data format is Valve's text-based VDF format, the kind used for localconfig.vdf and config.vdf (distinct from the binary formats used for appinfo.vdf and shortcuts.vdf). I am aware that it can be parsed and manipulated with other languages such as Python, but I have already manipulated the string. I simply need to update the file with the new string block.

Thanks!

Solution

  • If the file to be processed is small enough (I'd expect no problems up to 1MB, and significantly larger may be OK depending on the system) then Bash's built-in string substitution may be able to do what you need. This Shellcheck-clean code demonstrates the idea:

    #! /bin/bash -p

oldsec='"Section Name"
{
    "Key"       "{\"foo\":[\"bar\"],\"one\":\"two\"}"
    "AnotherValue"      "I am a string"
}'

newsec='"Section Name"
{
    "Key"       "{\"changed\":[\"value\"],\"three\":\"two\"}"
    "AnotherValue"      "Updated string"
}'

old_file_contents=$(< file.txt)
new_file_contents=${old_file_contents//"$oldsec"/"$newsec"}

printf '%s\n' "$new_file_contents" >file.txt