How do I deploy a power app component library from one environment to another?

I've created a component library in my environment, and I want to deploy it in a shared environment so others developing apps in that environment can use it. I tried exporting the components, then creating a component library in the shared environment and importing the components, but it tells me I need to use the get more components function (as if I were in an app referencing components) but I can't do that because the component library doesn't yet exist in the environment I'm creating in the app in. I need to deploy the library in the environment first, but how do I do that?

This is documented as a know limitation:

You can't import components from a locally saved component library. It's possible to save a component library using File > Save As > This Computer. But if you try to import such saved component library, you'll see the following error message:

Can you create a Power Platform solution with (only) the existing component library inside? So you can export the solution as a zip file, and import it in a different environment.