pythonyamlruamel.yaml

YAML / ruamel output formatting. Are there any other knobs?


I have the following python string dictionary and am using ruamel to dump it out as yaml file.

    import sys
    import json
    import ruamel.yaml


    dit = "'{p_d: '{a:3, what:3.6864e-05, s:lion, vec_mode:'{2.5, -2.9, 3.4, 5.6, -8.9, -5.67, 2, 2, 2, 2, 5.4, 2, 2, 6.545, 2, 2}, sst:'{c:-20, b:6, p:panther}}}"


    yaml_str = dit.replace('"', '').replace("'",'').replace(':', ': ').replace('{','[').replace('}',']')
    print(yaml_str)

    yaml = ruamel.yaml.YAML(typ='safe') # 
    yaml.default_flow_style = False
    yaml.allow_duplicate_keys = True
    data = yaml.load(yaml_str)
    print("data: {}".format(data))

    fileo = open("yamloutput.yaml", "w")
    yaml.dump(data, fileo)
    fileo.close()

The output it prints as:

    - p_d:
      - a: 3
      - what: 3.6864e-05
      - s: lion
      - vec_mode:
        - 2.5
        - -2.9
        - 3.4
        - 5.6
        - -8.9
        - -5.67
        - 2
        - 2
        - 2
        - 2
        - 5.4
        - 2
        - 2
        - 6.545
        - 2
        - 2
      - sst:
        - c: -20
        - b: 6
        - p: panther

but the expected / desired output is as follows:

    p_d:
      a: 3
      what: 3.6864e-05
      s: lion
      vec_mode:
        - 2.5
        - -2.9
        - 3.4
        - 5.6
        - -8.9
        - -5.67
        - 2
        - 2
        - 2
        - 2
        - 5.4
        - 2
        - 2
        - 6.545
        - 2
        - 2
      sst:
        c: -20
        b: 6
        p: panther

Is there any formatting that can be done/configured in ruamel to get this desired output i.e. instead of underscore use the proper tabs, and for the arrays alone use the underscores as represented?

Update to the Query: In the above query, in the dit string when I tried adding two sub strings like given below.

dit="'{p_d: '{a:3, what:3.6864e-05, s:lion, vec_mode:'{2.5, 1e9, -2.9, 3.4, 5.6, -8.9, -5.67, 2, 2, 2, 2, 5.4, 2, 2, 6.545, 2, 2}, sst:'{c:-20, b:6, p:panther}}}\n'{mgbp: '{ifftp:'{ipdp:'{ipdncf:'{rand_type:no, mfn:\"bbbb.txt\", rng_speed:-967901775, np:-1187634210}}}}}"

Getting the following error:

"ruamel.yaml.parser.ParserError: did not find expected <document start>"

I tried adding ---\n to the beginning of the string but still got the same error.

Any suggestions or comments ?


Solution

  • You convert the argument for the root level and for the root level key p_d into a list with your string replacement. So the output is exactly what you should expect to get in yamloutput.yaml.

    If that is what you want without affecting the value for key vec_mode you either have to do more specific parsing, or clean up after loading:

    import sys
    from pathlib import Path
    import ruamel.yaml
    
    path = Path('yamloutput.yaml')
    
    dit = "'{p_d: '{a:3, what:3.6864e-05, s:lion, vec_mode:'{2.5, -2.9, 3.4, 5.6, -8.9, -5.67, 2, 2, 2, 2, 5.4, 2, 2, 6.545, 2, 2}, sst:'{c:-20, b:6, p:panther}}}"
    
    
    yaml_str = dit.replace('"', '').replace("'",'').replace(':', ': ').replace('{','[').replace('}',']')
    
    def cleanup(d):
        if isinstance(d, list):
            ret_val = {}
            for elem in d:
                assert len(elem) == 1 and isinstance(elem, dict)
                ret_val.update(elem)
            return ret_val
        return d
        
    yaml = ruamel.yaml.YAML(typ='safe')
    yaml.default_flow_style = False
    data = yaml.load(yaml_str)
    data = cleanup(data)
    data['p_d'] = cleanup(data['p_d'])
    data['p_d']['sst'] = cleanup(data['p_d']['sst'])
    
    yaml.dump(data, path)
    
    sys.stdout.write(path.read_text())
    

    which gives:

    p_d:
      a: 3
      s: lion
      sst:
        b: 6
        c: -20
        p: panther
      vec_mode:
      - 2.5
      - -2.9
      - 3.4
      - 5.6
      - -8.9
      - -5.67
      - 2
      - 2
      - 2
      - 2
      - 5.4
      - 2
      - 2
      - 6.545
      - 2
      - 2
      what: 3.6864e-05
    

    If all lists consisting of a single element dict should be converted you can make cleanup call itself recursively.

    If the key ordering is important to you, you should not use typ='safe' and then (recursively) set the flow-style on the collection nodes to block-style.