pythonlistdictionary

How to create a dictionary with a key and a list of values from multiple folders and files


I have a three folders in GCS and I want to create a dictionary that has the folder name as the key and its contents as the values with without the file_type as the values. How do I do this?

For a bit more clarity, the folders are databases and I run

select * from information_schema

From there, I break down the files as needed and then place the csv files in the folders. After the folder and contents are created I send the dictionary to other teams.

Example: In the following folder I have these files temp_folder/stats/data_transfer/

extraction.csv
united_sync.csv
united_configs.csv
united_history.csv

In another folder temp_folder/stats/appointments/ I have the following

appointment_history.csv
appointment_configs.csv
appointment_transactions.csv

Expected Outcome:

folders_dict = {
    'data_transfer': ['extraction', 'united_sync', 'united_configs', 'united_history'],
    'appointment': ['appointment_history', 'appointment_configs', 'appointment_transactions']
}

Solution

  • How about something like this which reads files and put it in an list inside a dict:

    import os
    
    folders = {
        'appointment': 'temp_folder/stats/appointments/',
        'data_transfer': 'temp_folder/stats/data_transfer'
    }
    
    folder_contents = {}
    
    for folder_name, folder_path in folders.items():
        folder_contents[folder_name] = []
        files = os.listdir(folder_path)
        for file in files:
            name, extension = os.path.splitext(file)
            folder_contents[folder_name].append(name)
    
    print(folder_contents)