javamysqlmultithreadingspring-bootaws-aurora-serverless

How to fix "Lock wait timeout exceeded; try restarting transaction" for Mysql table when it was working before?


I have a Spring Boot application which asynchronously inserts data into a table:

ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(DAYS_IN_WEEK);


for (int i = 0; i < DAYS_IN_WEEK; ++i) {
    Runnable task = new MyThread(weekStartDate.plusDays(i));
    executor.execute(task);
}

executor.shutdown();

while (!executor.awaitTermination(1, TimeUnit.SECONDS)) {
    // wait until the processes in the executor will be terminated
}

log.info("Insertion is finished.");

Here is MyThread:

public class MyThread implements Runnable {

    // ..........

    public MyThread(LocalDate date) {
        // ..........
    }

    public void run() {
        log.info("date - {}", date.format(dateTimeFormatter));

        /*
          The similar script is being executed:
      
          "insert into my_table(col1, col2, ..., colN)" +
          "    (select col1, col2, ..., date" +
          "     from another_table " +
          "     where date = :date" +
          "    )";
        */
    }
}

The thing is that it was working perfectly throughout of months but now I get

2022-09-26 12:34:28.280 ERROR 5664 --- [pool-2-thread-3] o.h.engine.jdbc.spi.SqlExceptionHelper : Lock wait timeout exceeded; try restarting transaction 2022-09-26 12:36:58.529 ERROR 5664 --- [pool-2-thread-6] o.h.engine.jdbc.spi.SqlExceptionHelper : Lock wait timeout exceeded; try restarting transaction 2022-09-26 12:39:26.070 ERROR 5664 --- [pool-2-thread-4] o.h.engine.jdbc.spi.SqlExceptionHelper : Lock wait timeout exceeded; try restarting transaction 2022-09-26 12:45:34.697 ERROR 5664 --- [pool-2-thread-5] o.h.engine.jdbc.spi.SqlExceptionHelper : Lock wait timeout exceeded; try restarting transaction

exception. All I've searched for mention about nested transactions and locks on the same record or data, but the thing is that it was working before and also there cannot be record locks or modifications of the same record in different threads since each thread is executed on a data based on dates which can never be the same for different dates.

The database is Aurora Serverless 2.08.3 and there were no configuration changes (innodb_lock_wait_timeout, etc.) from my side (not sure if there is from AWS itself).

Also, SHOW ENGINE INNODB STATUS; doesn't show that there is a locked tables or rows. sudo killall -9 mysqld didn't help either.

So the questions is: What can be wrong with the application which was working 2 days ago perfectly with no incident?

So any help would be highly appreciated.


Solution

  • I found the problem.

    The thing was, that I was removed a redundant column in the entity class which described the table which the data should be selected from and inserted into the resulting table. Afterwards, I didn't removed the same column in the DB table.

    Most probably, that was somehow affecting on the locking mechanism in MySQL. Once I removed the corresponding column in the table, the locking issue was resolved.