loopscursorcommitbulk-collect

More efficient way to use cursor loop, insert into and commit for each row (Oracle)


I writed something like this below (it works), but in my case for 1,5 milions rows it is not so effective as I need (it will run maybe 2 days) I saw something like BULK COLLECT FETCH FORALL etc. but I am not managing to rewrite my code to this without errors. Can you help me with it?

Thank you

--It is my code for rewriting
    DECLARE
      cnt NUMBER;
      d_min NUMBER;
      d_max NUMBER;
      i NUMBER := 0;
      CURSOR ts_metatata_cur IS select * from (select rownum as rn, id_profile from ts_metadata where typ=7 and per=3600 order by id_profile) where rn between 1 and 100000;
    BEGIN
      for metadata_rec in ts_metatata_cur
      LOOP
            XTS.GET_PROFILE_AGGR(metadata_rec.id_profile, cnt, d_min, d_max);    --procedure with one IN parameter and three OUT parameter cnt, d_min, d_max
            Execute immediate 'insert into TMP_PROFILES_OVERVIEW (id_profile, cnt, d_min, d_max) values (' || metadata_rec.id_profile || ', ' || cnt || ', ' || d_min ||  ', ' || d_max ||')';
            i := i+1;
            if (i > 10000) then
              commit;
              i := 0;
            end if;
      END LOOP;
    commit;
    END;

If it is necessary I give here procedure which I call:

--this is procedure, which I call in my script
CREATE OR REPLACE PROCEDURE XTS.GET_PROFILE_AGGR(id_prof IN NUMBER, cnt OUT NUMBER, d_min OUT NUMBER, d_max OUT NUMBER)
AS 
  res varchar2(61);
BEGIN
  select cluster_table_name into res FROM XTS.TIME_SERIES TS where TS.id=id_prof;
  Execute immediate 'select nvl(count(*),0), nvl(min(time),0), nvl(max(time),0) from '|| res || ' where time_series_id=' || id_prof || ' ' into cnt, d_min, d_max;

EXCEPTION

  when others then
  null;

END;

Solution

  • I changed my approach to task after advices from another people. I INSERT INTO only grouped data instead of INSERT INTO each row.

    Something like this:

    CREATE OR REPLACE PROCEDURE XTS.GET_PROFILE_AGGR
    AS
    CURSOR time_series_cur IS select distinct cluster_table_name as res from xts.time_serie;

    BEGIN  
      for time_series_rec in time_series_cur  
      LOOP  
        Execute immediate 'INSERT INTO HDO.tmp_profiles_overview (id_profile, cnt, d_min, d_max) select time_series_id, count(*), min(time), max(time) from ' || time_series_rec.res || ' group by time_series_id';  
    
      END LOOP;  
    
    commit;   
    END;