I am looking to build a wordpress website with GatsbyJS (so no PHP usage. Just using Wordpress API and react JSX on the front-end).
I know I can get any core wordpress data at /wp-json route, but I'm not sure about plugin data. I've tried to look up plugin data and it does not seem to be included in wp-json.
I found this "compatible plugins" page: https://v2.wp-api.org/guide/plugins/
Does this mean that for most other plugins data would not be present in the Wordpress API?
Correct. Unless these plugins expose their data via the REST API (either by extending the existing endpoints or by creating their own, like some plugins do), you won't be able to access it.
In such cases, you either ask the developers to expose their plugin's data via the REST API or you code some custom endpoints yourself to get the data you need from them.