I have this proc sql step in SAS that takes about 1.7sec to complete, however, if i run this SQL via SQL server or any sql client, it takes only 0.04 sec to complete.
From the sastrace log i found that most of the time are spend on 'Total row fetch seconds', which doesnt really explain what cause the 3600% different between SAS Proc SQL and SQL client.
Summary Statistics for ODBC are:
Total row fetch seconds were: 1.736099
Total SQL execution seconds were: 0.000698
Total SQL prepare seconds were: 0.000168
Total SQL describe seconds were: 0.003049
Total seconds used by the ODBC ACCESS engine were 1.743318
What exactly is the 'Total row fetch seconds' and how can i improve the run time?
proc sql noprint;
create table work.out1 as
select a.search_key,
a.field1,
c.field2
from db.tabl_1 as a
join
db.tabl_2 as b
on a.search_key = b.search_key
and a.search_key in (&search_key)
join
db.Tabl_3 as c
on b.field2 = c.field2
and upcase(XXX_field) EQ "XXX";
quit;
&search_key = 19709, 19710, 19711, 19712, .........
Edit: to include Details log from sastrace
MPRINT(macro): proc sql noprint;
SYMBOLGEN: Macro variable SUBMISSION_STUDY_RUNS resolves to
19709, 19710, 19711, 19712, .........
MPRINT(macro): create table work.out1 as select a.search_key, a.field1, c.field2 from db.tabl_1 as a join db.tabl_2 as b on
a.search_key = b.search_key and a.search_key in
(19709, 19710, 19711, 19712, .........) join db.Tabl_3 as c on b.field2 = c.field2
and upcase(XXX_field) EQ "XXX";
SQLSRV: AUTOCOMMIT is NO for connection 11
SQLSRV: AUTOCOMMIT turned ON for connection id 11
SQLSRV_10181: Prepared: on connection 11
SELECT * FROM "db"."tabl_1"
SQLSRV: AUTOCOMMIT is NO for connection 12
SQLSRV: AUTOCOMMIT turned ON for connection id 12
SQLSRV_10182: Prepared: on connection 12
SELECT * FROM "db"."tabl_2"
SQLSRV: AUTOCOMMIT is NO for connection 13
SQLSRV: AUTOCOMMIT turned ON for connection id 13
SQLSRV_10183: Prepared: on connection 13
SELECT * FROM "db"."Tabl_3"
SQLSRV: AUTOCOMMIT is NO for connection 14
SQLSRV: AUTOCOMMIT turned ON for connection id 14
SQLSRV_10184: Prepared: on connection 14
select a."search_key", a."field1", c."field2" from "db"."tabl_1" a inner join "db"."tabl_2" b on a."search_key" = b."search_key" inner join
"db"."Tabl_3" c on b."field2" = c."field2" where ( a."search_key" in (19709, 19710, 19711, 19712, .........) ) and ({fn
UCASE(c."XXX_field")} = 'XXX')
Query i ran in the SQL client:
select a.search_key,
a.field1,
c.field2
from db.tabl_1 as a
join
db.tabl_2 as b
on a.search_key = b.search_key
and a.search_key in (19709, 19710, 19711, 19712, .........)
join
db.Tabl_3 as c
on b.field2 = c.field2
and XXX_field = 'XXX';
It seems like when SAS run proc sql, especially join, it will copy the whole table into SAS memory before doing the join/processing. The same query on SQL server would just join and return the rows that fit the join.
I have replaced all the query with SAS Data step, and the performance is now closer to what i get on the SQL server.
Please let me know if anyone have another work around/explanation