opensslbinaryfileshexdumpxxd

How to convert hexadecimal hash into binary at terminal?


Main question: How to use xxd -r?

Secondary details. This command outputs a hexadecimal hash:

openssl dgst -sha256 myFile |  awk '{print $2}' > myHashHex

I suppose that myHashHex is a "hexdump", isn't it?
I thought that xxd -r < myHashHex will convert... Well, it does not work.


NOTES

To test/compare. I think that this is the same (no headers) in binary:

openssl dgst -sha256 -binary myFile > myHashBin

So, I expected that xxd -r < myHashHex > newHashBin generates newHashBin=myHashBin.


PS: to test. The real life problem is to maintain all with sha256sum and only in one procedure convert it to double-sha256.

Instead, I use openssl dgst -sha256 -binary myFile| openssl dgst -sha256 > myDoubleHash
I would use the two-step procedure:

  1. (always produce myHashHex)
    openssl dgst -sha256 myFile | awk '{print $2}' > myHashHex

  2. (when need double) xxd -r myHashHex | openssl dgst -sha256 > myDoubleHash


Solution

  • Types of Hexdumps

    The problem is that xxd expects a formatted hexdump, but you are sending in a string of hex numbers. This is what xxd refers to as a "plain hexdump".

    A formatted hexdump looks like this (quote by Joseph Campbell):

    $ printf "Computers are like old testament gods; lots of rules and no mercy." | xxd
    00000000: 436f 6d70 7574 6572 7320 6172 6520 6c69  Computers are li
    00000010: 6b65 206f 6c64 2074 6573 7461 6d65 6e74  ke old testament
    00000020: 2067 6f64 733b 206c 6f74 7320 6f66 2072   gods; lots of r
    00000030: 756c 6573 2061 6e64 206e 6f20 6d65 7263  ules and no merc
    00000040: 792e                                     y.
    

    xxd will, by default, create this kind of hexdump.

    Use -p for hex strings

    Per the xxd manpage, -r expects to read a formatted hexdump:

    -r | -revert
        reverse operation: ... 
        Use the combination -r -p to read plain hexadecimal 
        dumps without line number information and without a particular column layout. 
        ...
    

    To reiterate, by default, xxd -r expects a formatted hexdump. To read or write a "plain hexdump" with xxd, use the -p option. Note that xxd expects flags to come before arguments, so if you end with a flag, you will get unexpected behavior.

    Verification

    If you can see that running openssl twice (initially with -binary) and running openssl with xxd produce the same result.

    $ openssl dgst -sha256 <( printf "Hello World!" ) |  awk '{print $2}' > myHashHex; 
    $ xxd -r -p myHashHex | openssl dgst -sha256 
    (stdin)= 61f417374f4400b47dcae1a8f402d4f4dacf455a0442a06aa455a447b0d4e170
    $ openssl dgst -sha256 -binary <( printf "Hello World!" ) | openssl dgst -sha256
    (stdin)= 61f417374f4400b47dcae1a8f402d4f4dacf455a0442a06aa455a447b0d4e170