MySQL RDS read replica service is not purging binlogs
We have a replica read database for MySQL (5.6.41) on AWS RDS service, which worked OK for the last 2 years, but it suddenly started to behave very differently during last 3 weeks: it uses space and is not normally returning it back. So I had to buy more storage for it to continue operating (you can see 2 peaks in the screenshot).
As I see the problem is that some daemon is automatically calling PURGE BINARY LOGS BEFORE 'mysql-bin-changelog.10xxxx'; but that log "10xxxx" is not deleted and just stays there. I've checked INNODB MONITOR OUTPUT, there is NO long time active transactions, show processlist shows NOTHING, yet ~100% CPU is used
space is not reclaiming! SHOW BINARY LOGS; shows >5200 records and this number continues to grow.
I've tried to close all incoming connections, even turned off "event scheduler" and replication process. Still the picture is not healthy:
CPU is just stuck at >50%! And there are NO sessions in DB (just rdsadmin@localhost)
Can you help me with the cause and how to recover? Because for now I have to buy ~50GB per 3-4 days for "nothing".
For now ANSWER: we've contacted AWS Support (paid developer support), but they did not find anything special and said that the problem is on their side in hardware probably. So we had to create new replica and migrate our custom schemas to it, killed the problematic one after that.
I will answer own question, because have not found any better solution, even after contacting and investigating with AWS Support.
We've contacted AWS Support (paid developer support), but they did not find anything special and said that the problem is on their side in hardware probably. So we had to create new replica and migrate our custom schemas to it, killed the problematic one after that.
- MySql can't make column auto_increment
- Insert auto increment primary key to existing table
- 3 mysql table join but not getting expected result that i want
- MySQL insert_id returning null even though there was an insert
- mysql select query within a serialized array
- MySQL: Large VARCHAR vs. TEXT?
- php_network_getaddresses: getaddrinfo failed error in Docker's adminer
- Correlated queries - In derived tables, can I have many nesting levels?
- Is there a MySQL option/feature to track history of changes to records?
- When is a database called as an Embedded database?
- Multiple foreign keys from one table to same field in another table in Laravel
- Datetime-value's time inserted incorrectly in one column but not in another MYSQL
- Syntax diagrams for MySQL
- Converting currency symbols '$' '£' to html code $ £?
- Create Index in mysql not working
- Incorrect string value: '\xEF\xBF\xBD' for column
- Understanding inner join in recursive CTE (MySQL)
- Update MySQL table with rank number of duplicates by name
- Mysql query for 3 tables & user meta
- Setting up new instance of MySQL what is Current Root Password?
- How to change the CHARACTER SET (and COLLATION) throughout a database?
- How to resolve circular foreign key constraints with NOT NULL in Hibernate
- mysql join to record first one which is not matched before
- Conditional statement in migration doesn't work laravel
- Mysql ERROR at line 1153: Unknown command '\'
- How do I find the command causing a replication error in MySQL
- Wordpress Meta Value (Serialized) Filter
- Can you access the auto increment value in MySQL within one statement?
- Can't connect to RDS instance from EC2 instance
- ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)