I have just seen on Firebase that a new feature has been released called In-App Messaging. Is there already an idea somewhere how to get this to work with Flutter?
No, you cannot use it with flutter yet. A flutter plugin needs to be created to be able to use the In-App Messaging API
You can check all flutter plugins that enable Flutter apps to use one or more Firebase services here:
https://github.com/FirebaseExtended/flutterfire
The flutter in-app messaging plugin is now available! You can find more information about in the following link: