I have more than 10,000,000 rows need to be inserted into one table. Considering the number of rows is large, I want to split the big table into 100 small table first, and then insert from table_001 to table_100. I expect to do these:
int main(){
int i = 0;
int j = 0;
int k = 0;
EXEC SQL BEGIN DECLARE SECTION;
char *buff[100] = { 0 };
EXEC SQL END DECLARE SECTION:
while( i < 100 ){
buff[i] = malloc(12);
sprintf(buff[i],"table_%3.3d",i+1);
i++;
}
i = 0;
while( i < 10000000 ){
EXEC SQL INSERT INTO :buff[j] VALUES(:v1,:v2);/* v1 and v2 is not important here, so I eliminate the code generate the v1 and v2 */
k++;
if( 1000000 == k ){
k = 0;
j++;
}
}
return 0;
}
But it seems that the table name can't be variable, is there a similar way to do these? Or is there any other good way to deal with big table in embedded SQL?
For a variable table name, you need to PREPARE
and EXECUTE
a dynamic SQL statement in which you've CONCAT
'd the table name into the statement.
Normally dynamic SQL built concatenated text is a bad idea as it is open to SQL injection. However, since you're not dealing with input from a user and it's the only way to handle a variable tablename you should be ok.
But I see no advantage to inserting the 10M rows into 100 smaller tables first, then from the smaller tables into the main one.
Just instert directly into the final table; commiting every 1000 rows or so; as you probably don't want 10M rows in a single commit.
You don't mention the platform and version of DB2 you are using, there may be bulk loading process available that would be an alternative.