bashfor-looprecursionzero-pad

Bash script to scan through folder and and delete zeros in each file


I have a folder consisting of 68 files (.txt files ) out of which 38 files have zeros starting from fourth row and ending in third column.

i would like to remove all the zeros from the files by using a bash script.

The .txt files are in following format ( i am attaching pictures of first three files in my directory ), you can see the for each file a new row with additional number of zeros was added.

Few examples of the files are listed below

regional_vol_GM_atlas1.txt

667869 667869.000000 
580083 580083.000000 
316133 316133.000000



regional_vol_GM_atlas2.txt

667869 667869.000000 
580083 580083.000000 
316133 316133.000000 
0 0.000000 
9020 9020.000000  
11065 11065.000000 



 regional_vol_GM_atlas3.txt

667869 667869.000000 
580083 580083.000000 
316133 316133.000000 
0 0.000000 
0 0.000000 
11651 11651.000000 

regional_vol_GM_atlas3.txt

667869 667869.000000 
580083 580083.000000 
316133 316133.000000 
0 0.000000 
0 0.000000 
0 0.000000 
12706 12706.000000 

for 38th file, 37 rows of zeros were padded starting from row three, how can i remove them through script ? the common file format of all the files in directory is regional_vol_GM_atlas*.txt


Solution

  • Remove lines like 0 0.0000 from the 3rd line AND trailing zeroes (and the dot) of last field like this:

    sed -i -e '3,$ {/0 0\.0\+/d;}' -e 's/\.0\+ *$//' regional*txt
    

    before:

    4536 23452345.0000
    0 0.0000
    

    after:

    4536 23452345
    

    (dot + 1 or more zeroes at the end of the line, changed by "nothing")

    warning: this is "in-place". Make a backup before trying this. It does not count how many columns are there. If this is a problem, then awk should be used.

    (make a test not in-place like this: sed -e '3,$ {/0 0\.0\+/d;}' -e 's/\.0\+ *$//' one_of_your_files.txt)

    Recursive case: if files are in several subdirs apply sed on the result of the find command:

    sed -e '3,$ {/0 0\.0\+/d;}' -e 's/\.0\+ *$//' $(find subdir_root -type f -name "regional*.txt")