linuxbashmd5sum

How to get the sha1 the sha256 the md5 on the same line?


i want to get the md5, the sha1 and the sh256 of all the file of my computer.

The expected out put is "the file name","the md5","the size".

main()
{
    liste=`sudo ls -R`
  for l in $liste
  do
    #echo $l
    g=`md5sum $l`
    printf "\"$l\","
    echo $g | awk '{printf("\"%s\",",$1)}'
    ls -lh $l | awk '{printf("\"%s\",",$5)}'
    printf "\n"
done
}
cd /
main

this is not working because it can't use md5sum in an other directory.

so I get this error message :

md5sum: rc6.d: No such file or directory
"rc6.d","",ls: cannot access 'rc6.d': No such file or directory

how do i get acces to the file ?

I tried :

#!/usr/bin/env bash
main()
{
    liste=`sudo find`
  for l in $liste
  do
    #echo $l
    g=`md5sum $l`
    printf "\"$l\","
    echo $g | awk '{printf("\"%s\",",$1)}'
    ls -lh $l | awk '{printf("\"%s\",",$5)}'
    printf "\n"
done
}
cd /
main

But i get this :

find: ‘./mnt/c/ProgramData/Microsoft/Windows NT/MSFax’: Permission denied
find: ‘./mnt/c/ProgramData/Packages’: Permission denied
find: ‘./mnt/c/ProgramData/VMware/VMware USB Arbitration Service’: Permission denied
find: ‘./mnt/c/ProgramData/WindowsHolographicDevices’: Permission denied
find: ‘./mnt/c/System Volume Information’: Permission denied
find: ‘./mnt/c/Users/cypri/AppData/Local/Packages/CanonicalGroupLimited.Ubuntu_79rhkp1fndgsc/LocalState/rootfs’: Permission denied
find: ‘./mnt/c/Users/cypri/AppData/Local/Packages/CanonicalGroupLimited.Ubuntu_79rhkp1fndgsc/LocalState/temp/{05418818-9381-4d3c-9934-ac417ee93067}’: Permission denied
find: ‘./mnt/c/Users/cypri/AppData/Local/Temp/WYU9188.tmp.dir’: Permission denied
find: ‘./mnt/c/Windows/appcompat/Programs’: Permission denied
find: ‘./mnt/c/Windows/CSC’: Permission denied

the best commande i fond so far is :

find -type f -readable -printf '%kkB ' -exec md5sum -- {} \;

how do i get the sha1 and the md5 on the same line : "sha1","md5",

i tried : find -type f -readable -printf '%kkB ' -exec md5sum -exec sha1 -- {} \; but it didn't work.


Solution

  • find / -type f -not \( -path '/dev/*' -or -path '/proc/*' -or -path '/sys/devices/*' \) -print0 |
      xargs -0 bash -c 'paste -d " " <(md5sum "$@") <(sha1sum "$@") <(sha256sum "$@") <(du -lh "$@")' bash |
      tr -s ' ' | cut -d ' ' -f 1,3,5,7-
    

    Here I use docker for demonstration:

    docker run --rm -i ubuntu:20.04 bash <<'SCRIPT'
    find / -type f -not \( -path '/dev/*' -or -path '/proc/*' -or -path '/sys/devices/*' \) -print0 |
      xargs -0 bash -c 'paste -d " " <(md5sum "$@") <(sha1sum "$@") <(sha256sum "$@") <(du -lh "$@")' bash |
      tr -s ' ' | cut -d ' ' -f 1,3,5,7-
    SCRIPT
    
    d41d8cd98f00b204e9800998ecf8427e da39a3ee5e6b4b0d3255bfef95601890afd80709 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0    /etc/subgid
    3aa8b92d1dd6ddf4daaedc019662f1dc cbd89fb1fa310fc4bc46866081454d5747922cd2 29128d49b590338131373ec431a59c0b5318330050aac9ac61d5098517ac9a25 4.0K /etc/bash.bashrc
    f9a1deea3a8fde4f992cc63ff939d923 c2a6d3cf400902bf8e4e97f84be440f607498e7c a49ab4136679f3fa4760385cb2e4e1c060afacf3c9e1a46d6be8717f59339184 4.0K /etc/apt/sources.list
    0081b49fa709cfd95827c75297b75ddd 76a4711ebfd6282a9fd9e8fffd8a83ca3fc66baa 4f4226614def131b475bcdaf1e5dda59a1af62fee838cbfc02a01c25e614efc1 4.0K /etc/apt/apt.conf.d/docker-autoremove-suggests
    ab6540f7278a05a4b7f9e58afcaa5f46 1a02e2f81f99a2fb621baf0cce0b332694982366 93e7e6d2fdb36b04cb10127e3b0d1b9d19d822327fd959484639bbbd65cce004 4.0K /etc/apt/apt.conf.d/01autoremove
    b02a49af378158c2c45158e991f9a987 2897d70bff6558284d8b6c7e806028ecca62d449 49d72455b2eaa50fa4b09b3ecffff753f3b90193266b2999f30e3f696403fde2 4.0K /etc/apt/apt.conf.d/docker-clean
    c69ce53f5f0755e5ac4441702e820505 43742ca9cbd8e8c18241a9b38aa302d92b0fa51c 364d5eeac5475b7dddfd629899ea88b91ed8d8e8e319c29bb9dbd6772e87ed55 4.0K /etc/apt/apt.conf.d/01-vendor-ubuntu
    7e9d09d5801a42b4926b736b8eeabb73 8d02d7c5507330294f8eba69adc413e35c70225b db749e19baf3b72ca2c157c70c52522cae23d94bc8b2dc5793fd43d427445367 4.0K /etc/apt/apt.conf.d/70debconf
    1caa06055a7a74a29c4476dccfb4726a 171bf894becc10c148b20d93056b2060ae113733 151044925acbe5c83434424d1b628e208790ecb2db0aabbe83dc8622495cf846 4.0K /etc/apt/apt.conf.d/docker-gzip-indexes
    341190f50b907d798bcb98c9e0d9cb07 abbf3fba2dbc78da2423167d3bfdda895a09e100 5970f921a86106d617c00deb6b2c7e5a37922fda8f2296bf43eebb83a72a2a4a 4.0K /etc/apt/apt.conf.d/docker-no-languages
    .
    .
    .
    
    0d5b70e555bab97581ee2b1d9943c976 11777fb50cde6629882c35ced448a4e0c91115dc 45e22485d11dde5533823fac333c06ed0b6a661672e4d6eef21ed31f34bb1696 68K  /usr/bin/iconv
    cb3a8469b0946d8012c0f8d7d051ffeb 1e04edc86fc20a35bbdb37865a492630727a1062 7f2e960e493c4586fbe6b11af0df148bdc93ead01e07598f281525df6edbef34 108K /usr/bin/du
    d8d3ce4d7f4b1e3ac3c3e7c9790f22ca 65324834933228e25e4a1b31cd7277214b6208ab 74af07c3d96e5fda903ac3df226b555ea6cad5d3b0aa250f3cea7713128bddf2 44K  /usr/bin/nohup
    bc5d246b4aedfd673f7c21fa7755a1cc 2336f2d23914d92d5c1c4019da797a7a8c24dda4 0c2134b4ee553b4fca4c011b5d5e6f0beb9394da7470cb925e5224196034e5e7 24K  /usr/bin/getopt
    40cc72ca80c0257ba6f5983af29fd589 f3f087097eab01d63aec5e500d82098eb563aac5 9b7c3f8459d41c2809f695040da541e17c057179c31c4f0be79a156104c8b633 52K  /usr/bin/sha1sum
    ac3b723b669e5d18ff9d9e5eecbf8869 f93e9b203d649d17dc1f18171abe6c64eff73b6d 6fbe7ade17043109f971ef60063700e92055e7c155210c2945b0851e96356dee 24K  /usr/bin/tload
    57b95ef16bac660b9d04b8fbfba65302 070fa89ff800e77eaece5bdf987b73f050d39488 71d27457a20148a13eda31216cd7bd56324a443295c8e15f958ed768f18061e4 16K  /usr/bin/locale-check
    cf277664b1771217d7006acdea006db1 17d380175c89fb145357edd7f1356f6274bfc762 34fbc467b8c624d92abcdf3edcf35ee46032618a6f23b210efab0e6824978126 4.0K /root/.bashrc
    d68ce7c7d7d2bb7d48aeb2f137b828e4 8e5d66ea938b5118633a4bd8c1d1e93376cd4e9d bbee58b1e0787bb851e7f7a4d0c187a8122d68eb67e5fa464696310398ac005b 4.0K /root/.profile
    d41d8cd98f00b204e9800998ecf8427e da39a3ee5e6b4b0d3255bfef95601890afd80709 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0    /.dockerenv
    

    How it works?

    1. You might want to exclude folders such as /dev and /proc because files like /dev/console and /dev/random would cause md5sum runs forever. Here we use find -not \( -path ... \) to skip those folder.

    2. find -print0 and xargs -0 help you handle filenames that contain white spaces.

    3. paste is useful to merge files line by line, for example:

      $ paste <(seq 3) <(seq 3)
      1       1
      2       2
      3       3
      
    4. du by default doesn't count links. We need to make sure du "$@" and md5sum "$@" generate the same number of lines by applying du -l so that each output has the same alignment for paste to process.

    5. The trailing bash in xargs bash -c '...' bash would be $0 when using -c flag. Though it can be any string but you should always make a sensible name for $0.