I have created a new ContentPart with a ContentPartRecord in a custom module and attached it to an existing content type on my website that already has hundreds of content items.
Now when I perform queries of the format contentManager.Query<MyPart, MyPartRecord>().List() I don't get any results since no MyPartRecords actually exist in the database yet.
Is there a way to make sure this happens for all content as soon as my part gets attached to a content type, or will I have to manually interact with all of the items before they become queryable?
This query is asking for all content items that have that part, which is not the case of items created before you added the part to the type definition. See the type definition as a blueprint more than a schema. Depending on what exactly you're trying to do, you may want to try to query by content type instead.
The part will get attached next time the item gets updated, pretty much. You could build something that scripts the operation on existing items, but nothing out of the box will do that.