I see an online environment for BigQuery but not for Cloud SQL; am I just missing it or should all queries be executed via cloud-sdk on CLI?
Yep, for now there is no UI for executing SQL queries in a Cloud SQL instance and you can use the Cloud SQL or maybe connect to the DB using MySQL Workbench for MySQL or any other third-party product (this also apply for the other DB flavors).