I have several dataframes I need to concatenate together. These dataframes contain results I got from APIs and did some edits on. All the dataframes have the same columns and all I really need to do is append them together. The problem I am facing is that the tables may not always exist depending on the original output with the APIs.
pd.concat(df1+df2+df3…) works how I want it to, but is it possible to dynamically generate the list of dataframes if they exist? The potential dataframes names will always fall within a set group, so I could iterate through a static list of ~12 potential names. The biggest problem I am having is not finding a way to do this but rather being able to use the list I produce in pd.concat(). So far, I’ve only been able to concatenate the list of table names into a data frame, rather than the data they contain. Is there a better way to think about this or a better function to use?
You can use the locals() (read more) function to get a list of the local variables in a function and access their values:
import pandas as pd
# List of DFs names
potential_names = ['df1', 'df2', 'df3', 'df4', 'df5', 'df6', 'df7', 'df8', 'df9', 'df10', 'df11', 'df12']
# Iterate through the DFs in the local scope thanks to locals()
existing_dfs = [locals()[name] for name in potential_names if name in locals()]
result = pd.concat(existing_dfs, ignore_index=True)