bashubuntu

Bash script to iterate files in directory and pattern match filenames


I need to process a large number of files in a directory. The files can be partitioned into several groups, based upon the file names. That is to say, the file names can be pattern matchedne which 'group' they belong to. For instance, the names are like this:

etc ...

Each 'group' has a different processing methodology (i.e. a different command is called for processing).

I want to write a bash script to:

  1. Iterate through all CSV files in the directory
  2. Determine which 'group' a file belongs to by pattern matching its name to known patterns (like the examples I gave above)
  3. Call a command based on the determined grouping.

I am running on Ubuntu 10.0.4. I am new to bash, and would appreciate skeleton code snippet that will help me get started in writing this script.


Solution

  • The easiest way is probably just to iterate each group separately. This side-steps the parsing issue entirely.

    DIRECTORY=.
    
    for i in "$DIRECTORY"/YYYYMMDD_*_bulk_import.csv; do
        # Process $i
    done
    
    for i in "$DIRECTORY"/YYYYMMDD_*_genstats_import.csv; do
        # Process $i
    done
    
    for i in "$DIRECTORY"/YYYYMMDD_*allstats.csv; do
        # Process $i
    done
    

    Set DIRECTORY to whatever directory you want to search. The default . will search the current working directory. Use quotes around "$DIRECTORY" in case the directory name contains spaces.