I'm using Sun Grid Engine to run a batch of jobs on Amazon Web Services EC2 nodes and I'd like to measure the wall time it takes to complete the whole batch. I'm fine with either from the time of submission to the time the queue is empty, or from the time the first job starts to the time the last job finishes (the times are long enough that the difference between the two is not important).
I tried using qacct, but looking at the walltime value in the output but it seems that this is the sum of the wall times for each job, rather than the wall time for all jobs together (in my use case each node can run multiple jobs concurrently). Is there a way that I can use qacct to get the wall time for the entire batch of jobs?
The accounting file contains submit, start and end times for all jobs. You should be able to grab the relevant jobs from there and pick the earliest start/submit and latest end time.