I need to iterate through all .rrd files inside a given directory and do fetch data inside each rrd database and do some actions and export them into single csv file in python script!
How can this be done in a efficient way? it's ok to advice me how to loop and access database data over multiple files?
I assume you have a rrdtool installation with python bindings already on your system. If not, here is an installation description.
Then, to loop over .rrd
files in a given directory and performing a fetch:
import os
import rrdtool
target_directory = "some/directory/with/rrdfiles"
rrd_files = [os.path.join(target_directory, f) for f in os.listdir(target_directory) if f.endswith('.rrd')]
data_container = []
for rrd_file in rrd_files:
data = rrdtool.fetch(rrd_file, 'AVERAGE' # or whichever CF you need
'--resolution', '200', # for example
) # and any other settings you need
data_container.append(data)
The parameter list follows rrdfetch.
Once you have whatever data you need inside the rrd_files
loop, you should accumulate it in a list of lists, with each sublist being one row of data. Writing them to a csv
is as easy as this:
import csv
# read the .rrd data as above into a 'data_container' variable
with open('my_data.csv', 'w', newline='') as csv_file:
rrd_writer = csv.writer(csv_file)
for row in data_container:
rrd_writer.writerow(row)
This should outline the general steps you have to follow, you will probably need to adapt them (the rrdfetch
in particular) to your case.