"Inability of MySQL to scale write requests beyond one node became a killer problem as data volumes grew leaps and bounds. MySQL’s monolithic architecture essentially forces application-level sharding.The development and operational complexity grows exponentially when the number of such instances grow from 1 to 100s and thereafter explode into 1000s."
So, If I use sharding for scaling up the application then
I cannot use the database itself for any cross-shard JOINs and transactions. It seems true but it Is it True ??
What's the better and efficient way to scaling the application as well as the database for say 10k concurrent requests and more. ?? I think going for microservices is not the solution here or is it ??
The answer is more nuanced than simply true or false. A horizontal sharding solution for MySQL like Vitess (vitess.io) does allow you to do both cross-shard joins and transactions, but just because you can doesn’t mean you should. A sharded architecture will perform best if you design it well and play to its strength, e.g. favoring single-shard targeted writes within any individual transaction. Enabling two-phase commit in Vitess to support cross-shard writes is possible, but will come at a significant performance cost. Whether that tradeoff is worth it differs from application to application, and generally speaking, adjusting the schema/workload is considered the better approach.
It’s better to think of microservices as a design principle than as a scaling trick. This architecture is more tailored to improving resilience and flexibility of deployment, by breaking up monolithic deployments into more loosely coupled, isolated elements. Because the complexity of managing resources for horizontal sharding align closely with the challenges of managing resources in a microservices architecture, Vitess is actually a great fit for a container orchestration environment and is commonly deployed/managed in containers using the Vitess Operator for Kubernetes.
In short, horizontally scaling MySQL is both feasible and commonly done, both in microservices architectures, as well as more traditional environments.
As an aside, 10K concurrent requests is relatively vague measure. It’s not abnormal for a well-configured single-server MySQL installation to serve hundreds of thousands of queries per second, so keep in mind that any scaling challenges you might face could also be resolved by optimizing your code, queries, schema and/or MySQL configuration.