pythonprogress-barpython-polarspolars

Show progress bar when reading files with globbing with polars

I have a folder with multiple Excel files.

I'm reading all of them in a single polars DataFrame concatenated vertically using globbing:

import polars as pl
df = pl.read_excel("folder/*.xlsx")

How can I have a progress bar that tracks files that are read?

Solution

  • You can’t get a progress bar directly from pl.read_excel("folder/*.xlsx") because Polars handles the glob internally. To show progress, you’ll need to loop over the files yourself, read them one by one, and concatenate them — then you can wrap the loop with something like tqdm.

    import polars as pl
import glob
from tqdm import tqdm

# Collect all Excel files
files = glob.glob("./*.xlsx")

# Read each file with a progress bar
dfs = []
for f in tqdm(files, desc="Reading Excel files"):
    dfs.append(pl.read_excel(f))

# Concatenate into a single DataFrame
df = pl.concat(dfs, how="vertical")